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Program: 

An  Outreach  to  Federal,  State,  Local  Regulators 

and 

Tribal  Governments 


November  1-2,  2016 

California’s  Department  of  Toxic  Substances  Control 

Cypress,  CA 


AGENDA 
Day  1 


0830-0900 

Welcome  &  Introduction  plus  Ice  Breaker 

John 

Scandura  & 

Roman  Racca 

0900-0950 

Purpose-  How  Did  We  Get  Here. ..A  30  year  Journey 

Jim  Austreng 
&  Doug 
Maddox 

0950-0955 

Mini-break 

0955-1035 

DoD  $14B  Liability,  DERP  Goals  and  Problem  with 
Statu  s  Qu  o 

Andy 

Schwartz 

1035-1045 

Break 

1045-1115 

ESTCP-  Demonstration  Program 

Bryan 

Harre 

1115-1215 

Lunch 

1215-1445 

The  Physics-  and  3-D  modeling  &  breaks  as  needed 

John  Jackson 

1445-1545 

Commercial  Application-  VSP  &  Vertical  CSM 

John  Jackson 
&  Andy 
Schwartz 

1545-1600 

Intro  To  QAPP 

Doug  Maddox 

1600-1630 

Open  discussion/Day's  Wrap  up 

All 

California  Department  of  Toxic  Substances  Control-  November  1-2 , 2016 


2 


AGENDA 
Day  2 


0830-0845 

Day  Two  Q&A 

All 

0845-1030 

QAPP  &  breaks  as  needed 

John  Jackson 

1030-1115 

QAPP  Exercise 

John  Jackson 

1115-1215 

Lunch 

1215-1300 

Accreditation,  Regulatory  Perspectives  and  3rd  Party  QA 

Andy  Schwartz  & 
Doug  Maddox 

1300-1415 

Risk  Management 

Andy  Schwartz 

1415-1425 

Break 

1425-1445 

VSP  Exercise 

All 

1445-1530 

Post  Remedy  Analysis  Exercise 

All 

1530-1600 

Summary 

Jim/Doug/John/Andy 

1600-1800 

Final  Exam 

All 
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Advanced  Geophysical  Classification 


How  we  got  Here  -  A  30  year  journey 


Jim  Austreng,  P.E. 

NAVFAC  Engineering  and  Expeditionary  Warfare  Center  (EXWC) 

Environmental  Restoration  Technology  Development  (EV31) 

1000  23rd  Ave 

Port  Hueneme,  CA  93043 

(805)  982-1660 
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Framework 


Interactive,  i.e.,  avoid  DBPP  (Death  By  Power 
Point) 

History  of  AGC  Development 

Understand  the  physics  supporting  AGC  (does 
not  mean  we  expect  you  to  be  a  geophysicist) 

Systematic  Planning  and  UFP-QAPP 

Limitations  of  AGC 

Quality  -  The  new  Paradigm 
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Key  Terminology  for  AGC 

Detection  Survey  or  Dynamic  Survey:  find  where  metal  exists  underground 
Cued  Survey:  Sitting  on  top  of  anomaly  to  collect  high  SNR  AGC  data 
Single-Axis  Sensor:  "Traditional"  metal  detector 
Multi-Axis  Sensor:  "New-Tech"  used  for  classification 
Electromagnetic  Induction  (EMI):  the  fundamental  basis  of  the  technology 

Anomaly:  Metallic  item  that  causes  a  geophysical  response 
Informed  Source  Selection:  Smart  anomaly  picking 
Clutter:  Non-hazardous  metal  "FRAGments"  or  other  debris 

Targets  of  Interest  (TOI):  potentially  hazardous  anomaly  or  any  other  item  of  interest 
Classify:  Determine  whether  "TOI"  or  "Non-TOI" 

Validate:  Prove  your  "classification"  worked  as  expected 

QC  &  Validation  Seeds:  Key  components  used  to  "validate"  the  AGC  process 
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Historical  Events 

1983  Tierrasanta  Accident-  not  the  only  one,  just  very 
influential 

1986  Superfund  Amendments  and  Reauthorization  Act/DERP 
1988  Base  Realignment  And  Closure  Act  (1st  Round) 

1990s  Proposed  Range  Rule  and  Range  Rule  Risk  Methodology 
1991  Develop  On  Site  Technology  (DOIT)  Committee 

1999  Interstate  Technology  and  Regulatory  Council  (ITRC)  UXO 
Workgroup 

2000  EPA  &  DOD  UXO  Management  Principles 
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Search  for  Technical  Solutions 


1991  Develop  On-site  Innovative  Technologies 
(DOIT  Committee) 

1994  U.S.  House  of  Representatives 
Subcommittee  Hearing 

1999  Inaugural  Session  of  ITRC  UXO  Work  Group 

2000  Fort  Ord  ODDS  Study 
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U.S.  House  of  Representatives 
Subcommittee  Hearing 


Members:  DoD,  DOI,  DOE,  EPA 
and  Western  Governors' 
Association 

DOIT  UXO  Sub-Committee  1994 

—  Push  for  Innovative  Technologies, 
e.g.,  3-dimensional  imaging 
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Progress 


1994-1999 

—  Jefferson  Proving 
Ground  Technology 
Demonstration 
Program,  Phase  I  -  IV 
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Eye  Opener 


•  JPG 

Examined  state  of  the  art  geophysical  platforms 
and  their  capability  to  "...detect,  classify  and 
remove  Unexploded  Ordnance  (UXO)." 

•  Phase  I  included  Congressional  mandate  to  test 
most  applicable  technologies  at  five  "Live  Sites" 

—  Results:  "The  Average  PD  [probability  of 
Detection]. ..was  0.44." 
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JPG  (Continued) 


^  •  PD  Improved  through  Phase  ll-lll  but  tradeoff 
was  increase  number  of  false  positives 


•  Phase  IV  (1996)  focused  on  demonstrating 
discrimination  capabilities 

—  Early  days  of  what  is  now  Advanced  Geophysical 
Classification 
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Analog 
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The  Answer? 
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Correctly  Processed 


HDaBB 

Q*  Q«a>  £?  Mm  to 
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1999 ITRC  UXO  Workgroup 


•  First  UXO  Report: 
December  2000  with 
emphasis  of 
demonstrating  digital 
technologies  and 
pursuing  those  with 
discrimination 
capabilities. 


Breaking  Barriers  to  the  Use  of  Innovative  Technologies: 
Suit’  Regulatory  Role  in  Unexploded  Ordnance  Detection  and 
Characterization  Technology  Selection 


December  2000 


Prepared  by 

Intent. lit1  Tertinnlirgy  and  Mt^ukvnry  Coii[iLMAilon  Work  timup 
iliu-Kplin.trd  Ordnance  WArk  I'rum 
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Defense  Science  Board 


Transmittal  Memorandum, 

William  Schneider,  Jr., 

October  24,  2003 

"The  Task  Force  found  that  technology 
can  be  of  dramatic  help  in  each  problem 
area.  The  current  cleanup  problem  is 
massive  in  scale  but  there  is  a  clear 
opportunity  to  save  tens  of  billions  of 
dollars  in  the  total  cleanup  process  by 
the  use  of  more  modern  technology" 


Report  of  the 

Defense  Science  Board  Task  Force 
on 

Unexploded  Ordnance 


November  2003 


Appendix  J,  page  2 

"We  then  "process  the  daylights"  out  of 
this  data  with  powerful  digitally 
implemented  algorithms  and  data 
processing  routines" 


Office  of  the  Under  Secretary  of  Defense 
For  Acquisition,  Technology ,  and  Logistics 
Washington ,  D.C.  20301-3140 
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Survey  of  Munitions  Response  Technologies 


Prepared  by: 

The  Environmental  Security  Technology  Certification  Program  (ESTCP) 
The  Interstate  Technology  &  Regulatory  Council  (TTRC) 

The  Strategic  Environmental  Research  and  Development  Program  (SERDP) 


June  2006 


The  publication  of  this  document  does  not  indicate  endorsement  by  the  Department 
of  Defense,  nor  should  the  contents  be  construed  as  reflecting  the  official  position  of 
that  Agency.  Mention  of  specific  product  names  or  vendors  does  not  constitute  an 
endorsement  by  any  of  the  authoring  organisations. 
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Sensors  of  Choice 


Wheeled  EMI  System 

Photo  courtesy  of  US  Army 


Typical  Electromagnetic  Induction 


Sensor 


Geometries  G-858  Portable  Cesium 
Magnetometer/Gradiometer 

Photo  courtesy  of  Geometries 
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We  Learned 


•  Dipole  Modeling 

•  Detection  Curves 

•  Geophysical  Prove  Outs 
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But  Wait! 
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Need  Geophysical  Prove-Outs? 


•  Does  not  translate  (or  indicate)  quality  of  field 
work 

•  Has  significant  statistical  uncertainties 

•  GPO  construction  is  not  representative  of  site 
conditions 

•  Excess  construction  costs,  needless  document 
production  requires  two  mob/de-mobs 
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Principal  of  GSV  Process 

«•  Provides  documentation  through  the  use  of  a  physics-based 
approach  that  is  transparent,  objective,  and  provides 
quantifiable  results  allowing  unbiased  validation  that  the 

?  project  data  quality  objectives  (DQOs)  and  hence,  the 
I  response  action  objectives  have  been  met 


RITS  2016:  QC/QA,  Risk  Management,  and  Explosive  Safety  Submissions 


QC&QA 


Paradigm  Shift 

Classification-  requires  High  Fidelity  Data 
Quality  from  Start  to  finish 
Accreditation 

Not  just  for  Classification-Quality  from  Start  to 
finish 
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MR  Response  Process 


Site  Preparation 

Vegetation  and  Surface  Clearance,  IVS,  Seeding 

I 

Geophysical  Data  Collection 

Dynamic  Detection  Survey,  Cued  Data  as  Needed 

I 

Analysis 

Parameter  Extraction,  Classification,  Initial  Ranked  Dig  List 

Dl 

Digging 

Training  and  Confirmation  Digs  until  Final  Ranked  Anomaly  List 

I 

Scoring 

Evaluate  Blind  Seeds,  Recovered  TOI,  Clutter  Rejection 
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Uniform  Federal  Policy  for  Quality 
Assurance  Project  Plans 

Jointly  developed  by  EPA,  DoD,  DOE 

Sets  requirements  for  all  environmental  data  collection, 
including  MR  data 

Provides  details  for 

—  Specific  data  requirements  or  other  information  that  must 
be  collected  to  demonstrate  conformance  to  requirements 

—  Required  data  in  37  worksheets 

—  Emphasis  on  systematic  planning 
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QAPP  Worksheet  for  Classification 


Worksheet  #  Title 


1  &  2 

Title  and  Approval  Page 

20 

Quality  Control  and  Corrective  Action 

3  &  5 

Project  Organization  and  QAPP  Distribution 

21 

Field  and  Data  Analysis  SOPs 

4,  7,  &  8 

Personnel  Qualifications  and  Sign-off  Sheets 

22 

Field  Equipment  Calibration,  Maintenance,  Testing  and 
Inspection 

6 

Communication  Pathways 

29 

Project  Documents  and  Records 

9 

Project  Planning  Session  Summary 

31 

Planned  Project  Assessment 

10 

Conceptual  Site  Model 

32 

Change  Control  Document 

11 

Data  Quality  Objectives 

33 

QC  Management  Reports 

12 

Measurement  Performance  Criteria 

37 

Data  Usability  Assessment 

13 

Secondary  Data  Uses  and  Limitations 

34 

SAP  Verification 

14&16 

Project  Tasks  and  Schedule 

35 

SAP  Validation 

17 

Sampling  Design  and  Project  Work  Flow 

36 

Product  QC  Tier  3  Summary  Report 
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Worksheet  Not  Used  for  Geophysical 

Classification 


Worksheet  # 

Title 

15 

Project  Action  Limits  and  Evaluation 

19 

Analytical  Methods/SOP  Requirements 

20 

Field  Quality  Control  Sample  Summary 

23 

Analytical  SOP  References 

24 

Analytical  Instrument  Calibration 

25 

Analytical  Instrument  &  Equipment 
Maintenance,  Testing  and  Inspection 

26 

Sample  Handling  System 

27 

Sample  Custody  Requirements 

28 

Laboratory  QC  Samples  Table 

30 

Analytical  Services  Table 
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QAPP  -  SOPs 


1 

Assemble  the  Geophysical  Platform  (MetalMapper,  etc.,)  and  verify  correct  operation 

2 

Test  Sensor  and  System  at  the  IVS 

3 

Production  Area  Seeding 

4 

Collect  Dynamic  Data  Using  the  advanced  geophysical  platform 

5 

Preprocess  Dynamic  Data  and  Identify  Anomalies 

6 

Collect  Static  Background  Measurements 

7 

Collect  Cued  Target  Measurements 

8 

Verify  Usability  of  Advanced  Sensor  Data 

9 

Background  Correct  Cued  Anomaly  Data 

10 

Invert  anomaly  data  to  extract  source  parameters 

11 

Compare  extracted  parameters  to  MEC  signatures  in  the  data  library 

12 

Develop  prioritized  dig  list  using  library  matching  and  other  factors 

13 

Verify  recovered  objects  are  compatible  with  predictions  based  on  the  advanced  sensor 
data 

14 

Develop  verification  sampling  dig  list  and  perform  verification  sampling 
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Quality  Throughout  the  Process 

•  Avoid  Garbage  in,  Garbage  out 


California  Department  of  Toxic  Substances  Control-  November  1-2 , 2016 


30 


Knowledge  Test  #1 


Classification  -  a  Paradigm  Shift  for  Munitions  Response 
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32 
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Nff.'rnC 

Naval  Facilities  Engineering  Command 

ENGINEERING  AND  EXPEDITIONARY  WARFARE  CENTER 


Back  Up  Slides 
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Courtesy  of  Les  Clarke,  Battelle 
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MILITARY  MUNITIONS 
RESPONSE  PROGRAM 
(MMRP) 

Advanced  Geophysical 
Classification  (AGC) 


US  Army  Corps 
of  Engineers. 


USACE  NATIONWIDE  CLEAN-UP  PROGRAM 


♦  FUDS  (Formerly  Used  Defense  Sites)  -  Mostlv  WWI.  WWII. 

•  BRAC  -  Current  Projects 

and  later  military  sites 

•  BRAC  -  Closed  Projects 

FUSRAP  (Formerly  Utilized  Sites  Remedial  Action  Program)  - 

♦  SUPERFUND  (National  Priorities  List)  - 

Low-level  radioactive,  non-DOD  sites,  including 

Former  Industrial  sites  with  hazardous 

DOE  Atomic  Energy  Era  projects 

waste;  USACE  work  for  EPA 

"Does  not  include  USACE  clean  up  work  at  Army,  AF,  NGB,  and  Reserve  Installations 


US  Army  Corps 
of  Engineers 


THE  REALLY  BIG  PICTURE 


DERP  Fiscal  Year  13  Annual  Report  To  Congress:  MMRP  CTC  ~$13B 
Fiscal  Year  17  ER(IRP  &  MMRP)  Budget  Request:  $1.3B 
Fiscal  Year  17  MMRP  Budget  Request:  $221. 1M 
($1 3B/$221 .1  M/year=59  years) 


J  I 1 - 1 1 1 1 1 - 1 1 - i 1 1 1 i 1 1 i 1 - 1 1 - i 1 1 —  i  i  i  i  i  i  ~ I- ~ ^ 

FY08  FY 1 1  FY14  FY17  FY20  FY23  FY26  FY29  FY32  FY35  FY38  FY41  FY44  FY47  FY50 

- Active  - BRAC  - FUDS 


Total  Number  Of 
DERP  Projects 

•  Army 

•  BRAC 

•  FUDS 
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of  Engineers 
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THE  REALLY  BIG  PICTURE  -  AT  THE  TOP 

DoD  Instruction  (DoDI)  4715.07,  Defense  Environmental 
Restoration  Program  (DERP),  May  21 , 2013 

DoD  Manual  (DoDM)  4715.20,  Defense  Environmental 
Restoration  Program  (DERP)  Management,  March  9, 

2012 
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US  Army  Corps 
of  Engineers 
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THE  REALLY  BIG  PICTURE  -  DERP  INSTRUCTION 

DoD  Instruction  (DoDI)  4715.07,  Defense  Environmental 
Restoration  Program  (DERP),  May  21 , 2013 

-  Establishes  policies,  assigns  responsibilities,  and  provides 
procedures  for  implementing  the  DERP 

-  Establishes  DoD’s  Interim  Risk  Management  (IRM)  policy 

•  Implement  IRM,  when  appropriate,  to  reduce  potentially  significant  threats  to 
human  health  at  sites  where  DoD  is  not  expected  to  conduct  an  investigation, 
removal  action,  or  remedial  action  for  an  extended  period  of  time 

-  Designates  the  Secretary  of  the  Army  as  lead  agent  for  the  FUDS 
program 

•  Army  identifies  FUDS  program  funding  requirements,  conducts  cleanup 
activities  at  eligible  properties,  and  reports  on  progress 

-  Establishes  the  DoD  Cleanup  Committee 

•  DoD  established  the  FUDS  Forum  Working  Group  under  the  DoD  Cleanup 
Committee 


US  Army  Corps 
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THE  REALLY  BIG  PICTURE  -  DERP  MANAGEMENT 

DoD  Manual  (DoDM)  4715.20,  Defense  Environmental 
Restoration  Program  (DERP)  Management,  March  9, 
2012 

-  Outlines  procedures  and  provides  further  guidance  on  the  policies 
in  DoDI  4715.07,  including: 

-  Environmental  Restoration  Account  Funding 

-  DERP  Eligibility 

-  Environmental  Liability  and  Cost-To-Complete  (CTC)  Management 

-  Federal  Facility  Agreements 

-  Response  Complete  (RC)  Determinations 

-  Green  and  Sustainable  Remediation 

-  Defense  State  Memorandum  of  Agreement  (DSMOA)  Eligibility 

-  Vapor  Intrusion 

-  Administrative  Record  Keeping 

-  5-Year  Reviews 

-  Restoration  Advisory  Boards 

-  Land  Use  Controls 
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THE  REALLY  BIG  PICTURE  -  DERP  MANAGEMENT 
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This  is  why  you  get  so  many  different  format! 
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QUICK  LOOK  AT  FUDS 
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A  NEW  FUDS  VISION: 

"RESPONSE  COMPLETE  IN  OUR  LIFETIME’’ 


FUDS  Guiding  Principles: 


1.  Goal  Focused:  Focus  our  efforts  on  leveraging  annual  funding  towards  achieving  DoD  IRP  goals  of  90% 
Response  Complete  by  the  end  of  FY18  and  95%  Response  Complete  by  the  end  of  FY21 . 

2.  Creativity:  Pursue  progressive  acquisition  strategies  to  address  multiple  FUDS  projects  and  phases  under 
coordinated  procurements  that  are  based  on  well-defined  and  achievable  objectives. 

3.  Innovation:  Effectively  assess  and  implement  advanced  geophysical  classification  at  all  phases  of  the  CERCLA 
process  at  FUDS  MMRP  projects  in  order  to  reduce  both  time  and  costs  for  munitions  cleanup. 

4.  Fiscal  Responsibility:  Plan,  resource,  and  execute  FUDS  projects  based  on  a  commitment  to  complete  the 
RI/FS  phase  of  work  to  within  four  to  five  years  of  initiation. 

5.  Commitment  to  Excellence:  Maximize  the  assignment  of  qualified,  trained,  and  experienced  FUDS  project 
management,  technical,  and  support  staff  to  serve  on  high  performing  virtual  PDTs  that  cross  traditional 
boundaries. 

6.  Continuous  Improvement:  Integrate  innovative  technology  and  optimization  processes  to  reduce  time  and 
costs  to  achieve  Response  Complete  for  ongoing  FUDS  projects  with  RA-C  and  RA-0  phases  with  durations 
greater  than  five  years. 

7.  Collaboration:  Elevate  regulatory  and  stakeholder  disagreements  and  delays  quickly  up  the  chain  of  command 
in  order  to  resolve  through  tiered  partnering  or  to  support  timely  decision  making  to  implement  response  actions 
under  our  DERP  authorities  even  when  regulatory  or  stakeholder  concurrence  cannot  be  achieved. 

8.  Transparency:  Share  information  and  findings  with  our  regulators,  stakeholders,  and  public  and  support 
collaborative  engagements  that  address  issues  and  uncertainties  as  part  of  the  decision-making  process. 
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LINKS  TO  FUDS  INFO 


•  DOD  DENIX  FUDS  Page: 

https://denix.osd.mil/fuds/overview/ 

•  USACE  FUDS  Home  Page: 

http://www.fuds.mil 

•  USACE  FUDS  Public  GIS  Page: 

http://maps.crrel.usace.armv.mil:7778/publicfuds/ 

•  USACE  FUDS  Forum  Community  Portal: 

https://www.fudsforum.org/ 


US  Army  Corps 
of  Engineers 


WHAT  IS  A  FUDS? 


Definition:  Real  property  that  was  under  the  jurisdiction  of 
the  Secretary  and  owned  bv.  leased  to.  or  otherwise 
possessed  bv  the  U.S.  and  those  real  properties  where 
accountability  rested  with  DoD  but  the  activities  at  the 
properties  were  conducted  by  contractors  that  were 
transferred  from  DoD  control  prior  to  October  1 7,  1 986. 

There  are  three  types  of  FUDS: 

•  Instalation  Restoration 

•  Building  Demolition/Building  Removal 

•  Military  Munitions  Response  Program 

Mission:  The  mission  for  the  FUDS  Program  is  to  employ  a  risk 
management  approach  to  perform  appropriate,  cost-effective 
cleanup  of  contamination  caused  by  the  Department  of  Defense  and 
to  protect  human  health,  safety,  and  the  environment. 
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CERCLA  PROCESS 


Sites  in  Progress 


investigation 


New  Sites 


Preliminarv  Assessment 

6 - *m 


Site  Inspection 

O - *■ 

Remedial  Investigation 

Q  »■ 


Record  of  Decision 


interim  Remedial  Actions  and 
Removal  Actions  may  occur  at  any 
time  during  the  CERCLA  process. 


Feasibility  Study 

o— 


Remedy  in  Place  (RIP)  is  an  important  milestone  in 
the  CERCLA  process*  At  this  point,  cleanup  systems  are 
constructed  and  operational. 


Remedial  Design 

O - — H 


Cleanup 


Remedy  in  Place 


Remedial  Action  Construction 

o —  ■ 


Response  Complete 


If  the  investigation  determines  cleanup  is  not  required,  or  w  hen 
cleanup  work  is  complete,  a  site  achieves  the  Response  Complete  (RC) 
milestone  (a  site  does  not  have  to  go  through  every  phase  to  achieve  RC). 


Remedial  Action  Operation 

O - 


Site  Closeout 


Site  Closeout  indicates  that  all  environmental  restoration  requirements  are  complete. 


Lone-Term  Management 

O- - 


Q  Start 


Milestone 


Complete 
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FUDS  STAKEHOLDERS 


LOCATION  OF  FUDS  SITES 

Total  Number  of  FUDS  Project:  5,144 
Note:  Territories  are  not  shown 
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As  of  30  Sept  2015 


FUDS  PROJECT  DISTRIBUTION 

(AS  OF  30  SEP  15) 


PRP/HTRW, 
211,  4% 


Total  FUDS 
5,144 


M  MRP/C  W 
M,  102,  2% 


PRP/MMRP, 
23, 1% 


PRP/HTRW, 
69,  4% 


FUDS  Projects 
Remaining 
1,852 


BD/DR,  375, 

MMRP/CW 

7% 

M,  17, 1% 

PRP/MMRP, 
14, 1% 


BD/DR,  39, 
2% 


CON/HTRW, 
131,  7% 


US  Army  Corps 
of  Engineers 


Remaining  FUDS  Projects 

Remaining  FUDS  Projects  not  at  RC:  1,852  [IRP  -  640/  MMRP  -  1,212] 


Northern  Mariana  Islands  1 1 
Palau  (PT)  0 

Puerto  Rico  (PR)  33 

U.S.  Virgin  Islands  (VI)  1 

American  Samoa  (AS)  6 

Guam  (GM)  9 


Astjjfco  SEP  201 5 

ES 
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ii  Him 

iC 


Y*  &  Beyond) 


(IRP  =  HTRW  +  CON/HTRW  +  BD/DR) 

$18.2  $17.9  $16.7  $14.1  $14.0  $13.2  $12.7  $12.9 


Fiscal  Year 

"Dollars  shown  reflect  CTC  reported  amounts  (not  adjusted  for  inflation) 


■  1 1 


US  Army  Corps 
of  Engineers 


UMilil'U 


OSD  GOALS  FOR  FUDS 


•  Installation  Restoration  Program  (IRP): 

>  Achieve  Response  Complete  at  90%  of  IRP  and  BD/DR 
sites  on  FUDS  by  the  end  of  FY2018 

>  Achieve  Response  Complete  at  95%  of  IRP  and  BD/DR 
sites  on  FUDS  by  the  end  of  FY2021 

•  Military  Munitions  Response  Program  (MMRP): 

>  Implement  Interim  Risk  Management  (IRM)  or  initiate  a 
Munitions  Response  action,  to  include  an  investigation  or 
a  removal/remedial  action  at  FUDS  MRS  that  have  not 
yet  reached  Response  Complete  for  90%  of  the  FUDS 
MRS  inventory  by  the  end  of  FY2018 
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FUDS  STATUS 


IRP  &  FY10  FY15  FY16  FY18  FY21 

BD/DR  Actual  Actual  Planned  Projected  Projected 

Sites 

Sites 

Remaining 

734 

571 

502 

325 

169 

Sites  at  RC 

73% 

80% 

82% 

89% 

94% 

MMRP 

Sites 

FY10 

Actual 

FY15 

Actual 

FY16 

Planned 

FY18 

Projected 

FY21 

Projected 

Sites 

Remaining 

1054 

1198 

1185 

1148 

1135 

Sites  at  RC 

38% 

42% 

42% 

44% 

45% 

Data  as  of  30  September  201 5 
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FUDS  PROPERTIES  MMRP 
COST  TO  COMPLETE  (CTC) 


■  EOY2015  CTC  $9.00B 

■  Net  Change  (+/-)  in  CTC  from 

EOY20 1 4  to  EOY20 1 5  +$70M 


So  Why  Advanced  Geophysical  Classification? 


US  Army  Corps 
of  Engineers 


r 


COMPETITION  TIME 


Need  Two  Volunteers 

•  Right  Side  /  Left  Side 

•  Good  in  Counting  Theory 

•  There  will  be  a  prize 


L 
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IN  2000,  BEFORE 
CLASSIFICATION  (BC) 


8 


>50 
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AND  THEN  THERE  WAS... 


ADVANCED  GEOPHYSICAL 
CLASSIFICATION 
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IN  2016,  AFTER 
CLASSIFICATION  (AC) 


ADVANCED  GEOPHYSICAL 
CLASSIFICATION  (AGC)  &  USACE 

■  AGC  Technologies  were  used  at  14  Munition 
Response  Sites  (MRSs)  at  9  properties  in  2015. 
Properties  have  been  mainly  in  the  Remedial 
Investigation/Feasibility  Study  (RI/FS)  phase  or 
Removal  Action  -  Construction  (RmA-C)  phase. 

►  Buckley  Field,  Camp  San  Luis  Obispo,  Culebra 
(4  Projects),  Marpi  Point  Field,  Mount  Owen  Rifle 
Range,  Raritan  Arsenal,  Scioto  Ordnance  Plant, 
Waikoloa  Maneuver  Area  (4  Projects),  Ft.  Ord 
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ESTCP  CLASSIFICATION 

DEMONSTRATION  SITES 
Our  Industry’s  Treatability  Study 


20  demonstration  sites 

Over  30  datasets  from  3  different  sensors 


■I" 
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BENEFITS  SUMMARY: 
THE  MMRP  AGC  EPIPHANY 

■  Cheaper 

►  Dig  MEC,  not  MD 

■  Better 

►  QC  /  QA  for  AGC 

►  Confidence  in  Decisions 

■  Faster 

►  Contracting  Consistency 

►  Performance  Consistency 

■  Greener 

►  Fewer  needless  digs 

►  Smaller  project  carbon  footprint 
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ADDITIONAL  TRAINING  AND 

RESOURCES 
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■  SERDP/ESTCP  website  contains  webcasts,  tutorials, 
and  demonstration  reports  on  the  classification 
technology 

►  www.serdp-estcp.org 

■  ESTCP  Technology  Transfer  Workshops 

■  M2S2  Webinar  Series 

►  http://www.clu-in.org/conf/tio/m2s2fv1 6-2  0421 1 6/ 

►  http://www.clu-in.org/conf/tio/m2s2fy1 5-2  02261 5/ 

►  http://www.clu-in.org/conf/tio/m2s2fv1 4-3  02251 4/ 

►  http://www.clu-in.org/conf/tio/m2s2-3  04221 3/ 


■  Interstate  Technology  Regulatory  Council  (ITRC)  Geophysical  Classification 
For  Munitions  Response:  http://www.itrcweb.org/gcmr-2/ 

►  Archived  web  training:  https://clu-in.org/conf/itrc/GCMR  01 1 41 6/ 
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Topic  3  -  Summary  of  ESTCP 
Demonstrations  2007  to  Now 


Bryan  Harre 
NAVFAC  EXWC 
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Enabling  Objectives 


■  Identify  investigation/removal  site 
types  where  advanced  geophysical 
classification  was  performed 

■  Discuss  munitions  types  and 
combinations  of  munitions  used  at 
the  demonstration  sites 

■  Describe  the  anomaly  density  and 
geologic  noise  that  can  impact 
classification  performance 

■  Explain  the  cost  savings  that 
advanced  geophysical  classification 
can  provide 


Summary  Report:  ESTCP  Demonstrations  of  Advanced  Geophysical 
Classification  for  Munitions  Response 

October  2015 
Version  2 
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ESTCP  Demonstration  Sites 
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Summary  of  Demonstration  Sites 


Degree  of 
Difficulty 

Characteristics 

Example 

Demonstration 

Expectation 

Easy 

Limited  munitions  types,  low 
anomaly  density,  terrain  and 
vegetation  allow  for  high  quality 
data  collection,  no  geologic 
interference 

Pole  Mountain,  WY 

Camp  George  West, 

CO 

SW  Proving  Ground, 

AR 

Almost  all  analysts 
correctly  classify  all 
seeds,  eliminate  -90% 
of  the  clutter  from  the 
final  dig  list 

Typical 

Mixed  munitions,  none  smaller  than 
37-mm,  low  to  moderate  anomaly 
density, ,  terrain  and  vegetation 
allow  for  high  quality  data 
collection,  moderate  geology 

Spencer  Range,  TN 
Camp  Beale,  CA 

Most  analysts  correctly 
classify  all  seeds, 
eliminate  -70%  of  the 
clutter  from  the  final  dig 
list 

Difficult 

Mixed  munitions,  smaller  than  37- 
mm  TOI,  high  anomaly  anomaly 
density, ,  conditions  make  data 
collection  challenging,  complex 
geology 

Massachusetts  Military 
Reservation,  MA 

Even  the  most  skilled 
analysts  may  not  detect 
all  TOI,  eliminate  -50- 
70%  of  the  clutter  from 
the  final  dig  list 

Expectations  for  Easy,  Typical,  and 
Hard  Sites 
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Demonstrator  Results 


■  Not  all  demonstrators 
are  equal 

♦  Major  reason  DoD 
has  implemented  the 
DoD  Advanced 
Geophysical 
Classification 
Accreditation 
Program  (DAGCAP) 


Photo  courtesy  of  ESTCP. 
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Munitions  Types  and  Combinations  of 

Munitions 


■  Larger  munitions  are  easier  to  classify 
than  smaller  munitions  and 
classification  is  easier  on  sites  with 
fewer  different  types  of  munitions. 

■  Smaller  37mm  look  more  similar  to  the 
clutter  to  the  sensor,  particularly  the 
fragments  left  by  the  detonation  of 
larger  items 

■  20mm  look  even  more  like  clutter  and 
will  impact  the  economics  of  using 
classification 

■  37mm  projectiles  combined  with  other 
larger  munitions  are  routine  to  classify 


ISO80  and  37mm  Projectile.  Photo 
courtesy  of  ESTCP. 
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Anomaly  Density 


Detection 
survey  typically 
used  by  project 
team  to  identify 
areas  too 
dense  for 
classification 

■  Portions  of  site 
may  require  a 
different 
approach,  i.e. 
mag  and  flag 
or  sifting 
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Geologic  Background  Variation  (1) 


X  <*647265  011 


Graphic  courtesy 
of  ESTCP. 
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Geologic  Background  Variation  at  the 
former  Camp  Beale 
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Geologic  Background  Variation  (2) 


Geologic  Background  Variation  at 
Waikaloa 


Graphic  courtesy 
of  ESTCP. 
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Target  of  Interest  (TOI)  Signature 

Library 


HF 


File  Window  lools  Help 


&  cj  @HH 


Recent  Files 


C:\Users\p003502B\Documents\MR-201424\ESTCP  Library  -  HD\HDF5  Examples  from  Herb\IDLwork\Lib_HDF5Files\MM\4.2in_M329A2_N00173-402_8ms_Library.h5 


Item  Meta  Data  (28800, 2) 


Group  size  =  1 
Number  of  attributes  =  13 
Class  =  Projectile 
Comments  =  Smoke 
Common  Name  =  4.2in 
Condition  =  fired/Pristine 
Dimensions  (OD,LKmm)=  106,435 
Fins?  =  Y 
Fuse?  =  N 
Mark/Mod  =  M329A2 
Name  =  4.2in 
Qualifier/Pedigree  =  A 
Rotating  Band?  =  Y 
Serial  Number  =  N00173-402 
Spotting  Charge?  =  N 


Collection  of 
TOI  signatures: 

1.  Metadata, 

2.  Sensor  data 

3.  Polarizations 

ESTCP  generated 
DOD  maintained 
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Unexpected  Munitions 


Beale  -  Digging  uncovered  a  number  of  fuze 
parts  that  had  not  previously  been  identified. 

Explosives  safety  experts  determined  that 
these  fuze  parts  were  not  hazardous,  and  not 
a  TOI. 

■  Fort  Sill  -  Presence  of  six  “hollow  steel  ball 
assemblies,”  which  are  hazardous 
components  of  the  40-mm  grenades  known  to 
be  present  on  the  site. 

■  Involve  Explosives  Safety  Specialists  early  in  Non-hazardous  clutter 

the  planning  process,  and  transmit  knowledge 

of  hazardous  components  to  the  analysts. 

■  Classification  is  robust  to  the  presence  of 
unexpected  munitions  on  the  site  and 
procedures  to  process  the  data  are  now 
standardized. 


T-Bar  Fuze 
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Munitions  Size 


Classification  can  identify  the  approximate  size  of  munition 

■  Significant  Benefit  in  reducing  the  Explosives  Safety  Quantity 
Distance  Arc 


60 
50 
40 
30 
20 
10 
0 

Small  Medium  Large  Small  Medium  Large 


Spencer  Range 


Camp  Ellis 


□  Small 
■  Medium 
Large 


Southwestern  Proving  Ground 


Small  Medium  Large 


Actual  TO  I  Size 


Actual  TOI  Size 


Actual  TOI  Size 


Classification  by  size. 
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Vehicle-towed 


Receive  coils 


Transmit  coil 


Advanced  EMI  Sensor  developed  by 

ESTCP 


Multiple  coils  measure  the  complete  response  of  buried 
items  (spatially  and  temporally) 


Person-portable 


Cart-mounted 


Advanced  Geophysical  Classification 

Sensors 


Sensor 

Description 

Applicability 

MetalMapper 

•  1 -meter  cube 

•  3-axis  transmit 

•  Seven  3-axis  1 0-cm  receive  cubes 

•  Continuously  samples  to  8  ms  after  transmitter 
turn-off 

•  Dynamic  and  cued  modes 

•  Requires  vehicle  to  maneuver 

•  Require  GPS 

•  Depth  capability  similar  to  EM61  - 
comparable  transmit  moment 

•  Commercially  available 

MetalMapper 

2X2 

•  Four  35-cm  transmitters 

•  Four  8-cm,  3-axis  receive  coils  centered  in  each 
transmitter 

•  Overall  dimension  80cm  square 

•  Backpack  1 5  pounds 

•  Continuously  samples  to  25  ms  after  transmitter 
turn  off 

•  Dynamic  and  cued  modes 

•  Cart  and  litter-carried  deployment 

•  Does  not  require  GPS,  but  available 

•  Will  have  less  depth  capability  due  to 
smaller  transmit  moment 

•  Commercially  available 

MPV 

•  50-cm  diameter  transmitter  -  one  dimension  only 

•  Five  3-axis  8cm  receiver  cubes 

•  Can  be  manipulated  in  3D  to  get  multiple  looks  at 
the  target 

•  Carried  on  a  wand,  1 2  pounds 

•  Continuously  samples  to  8  ms  after  transmitter 
turn  off 

•  Dynamic  and  cued  modes 

•  Small  and  maneuverable  for  wooded 
areas  and  challenging  terrain 

•  Does  not  require  GPS  to  operate 

•  Uses  locating  beacon 

•  Will  have  less  depth  capability  due  to 
smaller  transmit  moment 

•  Can  be  contracted  through  developer 

Topic  3  -  Revised  1 0/1 6 


14 


Dynamic  Surveys  with  Advanced 

Sensors 

■  Advanced  sensors  for  detection  produce  data  that  has 
higher  spatial  resolution  and  allow  the  use  the 
advanced  sensor  data  to  make  smarter  anomaly 
selection  decisions 

■  Dynamic  data 

♦  Informed  Source  Selection,  a  filtering  step  is  used 
to  identify  only  those  anomalies  that  could  have 
been  caused  by  a  metal  object  with  the 
characteristics  of  a  TOI,  for  example  one  that  is 
well-modeled  by  a  dipole. 

♦  Make  classification  decisions  using  intrinsic  target 
parameters  extracted  from  the  dynamic  data  only. 


Topic  3  -  Revised  1 0/1 6 
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Lessons  Learned  -  Applicable  at  Most 

Sites  But  Not  All 


ESTCP-sponsored  demonstrations 
at  over  25  sites 


Maybe  Not  Applicable 

•  Anomaly  densities  over  ~4000 
per  acre 

•  Extensive  geologic  background 

•  Landfills  and  disposal  pits 

•  OB/OD  areas 

•  Almost  all  items  are  TOI 


Topic  3  -  Revised  1 0/1 6 
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Potential  Savings  on  a  100-acre  Remediation 


Current  Practice 


80%  Reduction 
of  Clutter 


45%  Savings 


1=1 


Mob/Demob 
Surface  Sweep 
IVS  &  Seeds 
Detection  Survey 
Cued  Data 
Dig  UXO  &  Seeds 
Dig  Clutter 


Topic  3  -  Revised  1 0/1 6 
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Fun  Facts  About  the  Sites 


■  There  are  few  native  TOI  on  most  sites.  Their 
abundance  ranged  from  0.04%  of  the  total  1 1  number 
of  items  dug  at  SWPG  to  1 .5%  at  Camp  George  West. 

■  Depth  distribution  of  the  native  clutter  is  very  similar  for 
wide  range  of  sites.  On  all  six  example  sites,  the  depth 
distribution  peaks  in  the  top  10  cm  and  on  four  of  the 
six,  the  vast  majority  of  clutter  was  in  the  top  20  cm. 

■  Seed  plans  for  the  demonstrations,  which  were  meant 
to  put  seeds  at  representative  depths,  were  in  fact 
conservative.  Most  of  the  TOI  were  found  at  depths 
shallower  than  the  seeds  were  emplaced. 


Topic  3  -  Revised  1 0/1 6 
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Additional  Training  and  Resources 

■  SERDP/ESTCP  website  contains  webcasts, 
tutorials,  and  demonstration 
reports  on  the  classification 
technology 

♦  www.serdp-estcp.org 

■  ITRC  has  web  document  and 
online  training 


Topic  3  -  Revised  1 0/1 6 


19 


Summary 


Commercially  available  sensors  -  MetalMapper  and  MetalMapper 
2x2  are  available  for  sale  or  rental 

■  Analysis  Software:  The  UX  Analyze  classification  software  has 
been  incorporated  into  the  GeoSoft  Oasis  Montaj  platform 

■  Signature  Library:  ESTCP  has  assembled  a  comprehensive 
library  of  signatures  of  common  munitions 

■  Quality  Assurance  Project  Plan:  The  Environmental  Data  Quality 
Work  Group,  supported  by  ESTCP,  has  developed  a  QAPP 
template 

■  Experienced  contractors:  1 0  companies  have  participated  in  the 
ESTCP  demonstrations 

■  Accreditation  Program:  DoD  has  mandated  an  accreditation 
program  for  contractors 

■  Informed  regulators:  ESTCP  has  incorporated  the  input  of 
regulators  throughout  the  demonstration  program 


Topic  3  -  Revised  1 0/1 6 
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Questions 


Navy  Training 
Advanced  Geophysical 
Classification 


John  M.  Jackson 

Geophysicist 
USACE-  SPK/EMCX 


US  Army  Corps  of  Engineers 

BUILDING  STRONG 


Outline 


■  Why  Classification 

■  How  it  Works 

■  Example  Workflow  and  QC/QA  -  Former 
Camp  San  Luis  Obispo  Example 

■  Summary 


Acknowledgements 

►  SERDP  &  ESTCP 

►  Camp  San  Luis  Obispo  PDT 

•  CESPL,  California  EPA,  ESTCP,  CH2M  Hill 


BUILDING  STRONG, 


Costs  Are  Dominated  By  Digging  Scrap 


Often  <1%  are  UXO 

100 

- 1 - 1 - 1 — 

- 1 - 

Example:  Camp 
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Site  '  Survey  and  '  Vegetation  '  Scrap  Metal'  UXO 

Assessment  Mapping  Removal  1 

Removal  Removal 

BUILDING  STRONG, 


What  Is  Classification? 


Attempts  to  assign  an  input  value  to  one  of  a 
given  set  of  classes  based  on  some  attributes 
of  the  input 


a  la  Wikipedia 


►  Is  the  incoming  e-mail  “spam”  or  “not-spam”? 

►  Is  the  buried  object  that  caused  this  anomaly 
“UXO”  or  “not-UXO”? 


BUILDING  STRONG, 


How  Do  We  Classify  Munitions? 


BUILDING  STRONG, 


If  It’s  Going  To  Work... 


■  It’s  got  to  be  a  principled,  data-based 
approach  to  classify  targets  as  either  “non- 
hazardous”  or  “targets  of  interest” 


BUILDING  STRONG, 


How  It  Works 


z 


BUILDING  STRONG 


Echolocation:  a  Familiar  Analogy 


ft 


of  sound  tell  size, 
shape  and  distance 


Interesting  fact:  what 
causes  the  sound 
reflection? 


\ 

\ 


►  Acoustic  waves... locating  objects  by  reflected  sound. 
Used  by  dolphins  and  bats... 
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Electromagnetic  Induction  Sensors  -  Review 


Typical  Electromagnetic  Induction  Sensor 


Excitation  Pulse 


Classification  Basics 
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Electromagnetic  Induction  Sensors  (cont.) 


Typical  Electromagnetic  Induction  Sensor 


Induced  Target 
Response 
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Electromagnetic  Induction  Sensors  (cont.) 


Typical  Electromagnetic  Induction  Sensor 


Secondary 

{Induced) 

Field 


Induced  Flux 


Transmit  Coil 


Induced 

Dipole 

Moment 


Receive  Coil 


Sense  Induced 
Field 
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We’ve  seen  how  the  methods  are 
similar,  here  is  one  major  difference... 


Distance:  Using  echolocation,  Dolphins  can  detect  a  2.54cm  sphere  at  73m! 

Your  chance  to  shine... 

At  what  distance  can  EMI  methods  detect  a  2.54cm  metallic  sphere? 

Answer:  0.4m 

Detection  Distance  of  a  1  -inch  Sphere 

EMI 


0  10  20  30  40  50  60  70  80 


EMI  response  is  inversely  proportional  to  the  Distanced  (sensor  to  object). 

The  sensors  have  to  be  very  close  to  the  object  in  order  to  detect  and 
characterize  buried  objects! 


BUILDING  STRONG, 


A  Term  You  Need  To  Know: 

Polarizability 

■  A  set  of  values  that  fully  describe  the  decay  of 
electromagnetic  fields  that  have  been  induced  in  an 
object  by  an  external  EM  source,  after  the  external  EM 
source  is  removed.  Polarizability  values  exist  for  each  of 
the  object’s  three  principal  axes 

■  They  are  used  here: 

►  The  EMI  Dipole  Model 

Y< 1 )  =  (xOnRnTIoCR'C 

The  EM  Sensor  properties 

Signal  We  &  Geometry  of  the 

Measure  sensor  wrt  target 


B(t) 


Target  properties 
(polarizabilities) 
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BUILDING  STRONG, 


Principal  Axis  Responses 


Normalized  response  (polarizability)  for  excitation  in  object’s 
principal  axis  directions  are  the  fundamental  EMI  attributes 

UXO  items  are 
symmetrical,  so  two 
of  the  principal  axis 
responses  are  the 
same 

Irregular  clutter  items 
have  three  different 
principal  axis  responses 


Key 

Point 


Munitions  are  symmetrical,  which  is  used  to 

identify  them 


BUILDING  STRONG, 


A  Simple  Analogy 

Name  That  Object... 


A  Simple  Analogy 


Name  That  Object... 


Top  View 
(compare  to  “Top 
View 

Polarizability”) 


Front  View 
(“Front  View 
Polarizability”) 


Side  View 

(“Side  View  Polarizability”) 


ifi 


BUILDING  STRONG, 


Multiple  Measurements  are  Required  to 
Completely  Characterize  a  Target  with  a 
Single-Axis  Sensor  (cont.) 


At  each  position  the  field  lines  only  intersect  the  target  in 

one  direction 


BUILDING  STRONG, 


Multiple  Measurements  are  Required  to 
Completely  Characterize  a  Target  with  a 
Single-Axis  Sensor  (cont.) 


At  each  position  the  field  lines  only  intersect  the  target  in 

one  direction 


BUILDING  STRONG, 


Multiple  Measurements  are  Required  to 
Completely  Characterize  a  Target  with  a 
Single-Axis  Sensor  (cont.) 


At  each  position  the  field  lines  only  intersect  the  target  in 

one  direction 


BUILDING  STRONG, 


The  Trick  Is  Illuminating  Each  Axis 

With  a  Tx  Pulse 


This  is  what  Advanced  EMI  Sensors  look 

like 


►  Multiple  coils  measure  the  complete  response  of  buried 
items  (spatially  and  temporally) 


Advanced  EMI  Sensors:  designed  for 

UXO  classification 
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ffl 

Multiple  transmitters  and  receivers  are  used  to  fully 

‘light  up’  or  illuminate  the  object 
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Measured  data  are  affected  by  burial 
depth  and  object  orientation 


BUILDING  STRONG** 
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Measured  data  are  affected  by  burial 
depth  and  object  orientation 
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BUILDING  STRONG, 


Polarizabilities  do  not  change  with  burial 

depth  or  orientation 


Advanced  Classification  Is  Inherently  A  Three-Step 

Process 

1 .  Collect  Field  Data 

►  Specialized,  classification-specific  instruments 

2.  Get  fingerprint  for  each  detected  anomaly 

►  Field  Data 

►  Specialized  Software 

►  Trained  Analyst 

3.  Assess  if  the  fingerprint  is  something  we  need 

to  be  concerned  about 

►  Polarizabilities 

►  Library  fingerprints  of  things  I’m  concerned  about 

►  Specialized  Software 

►  Trained  Analyst 
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BUILDING  STRONG, 


How  Do  We  Get  Those  Fingerprints? 


■  Another  Term  you  need  to  know: 

‘Geophyiscal  Inversion’ 

►  a.k.a.  Backward  model! 
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BUILDING  STRONG, 


This  Is  A  Forward  Model 


Given  this: 


y=6x2+4x-8 


For  each  value  of  x 
we  can  calculate  y 


X 

y 
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3.5 

95.5 
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4.5 

147.5 

5 

178 

And  we  can  graph  it 


4/18/2017 
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BUILDING  STRONG, 


This  Is  A  Backward  Model 
(aka  Inversion) 


Given  this: 


We  can  make 
educated  guesses 
on  the  formula... 


- Given 

- y=6xA2+4x+8 


Until  we  get  it 
correct 


4/18/2017 
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BUILDING  STRONG, 


Receiver  Receiver 


From  Field  Data  to  Predictions 
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BUILDING  STRONG, 


From  Field  Data  to  Predictions 


Field  Data 
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m 

Once  We  Have  The  Fingerprint... We  Classify! 


■  Classification  via  template  matching 
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Polarizability  Examples  “EMI  Fingerprints” 
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BUILDING  STRONG, 


Polarizabilities  completely  specify  the 
target’s  EMI  response  characteristics 


Object  Property 

Polarizability 

Property 

Cylindrical  Shape 

Axial  Symmetry 

Wall  Thickness 

Decay  Rate 

Physical  Size 

Magnitude 

BUILDING  STRONG, 


What  If... 


■  Someone  thinks  there’s  a  UXO  out  there 
that’s  not  in  my  library? 

►  First:  make  sure  it  makes  sense 

►  Then 

•  Add  it  if  you  can 

•  Look  for  obvious  traits 


...Big,  Ugly  &  Scary 

(i.e.  large,  axial  symmetry  and  thick  walled) 
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BUILDING  STRONG, 


What  If... 


►  There  are  groups  of  things  with  the  same  fingerprint  that  I 
don’t  know  about? 

We  call  this:  Analyze  for  clusters 

How  it  works: 

1  -  Select  one  of  the  inversion  solutions  to  be  the  basis  of  the 
analysis 

2-  Rack  &  Stack  every  other  inversion  solution 

3-  Compare  each  of  (2)  to  (1),  one  by  one 

If  (2)  similar  to  (1 )  then  add  to  the  (1 )  group 

This  is  just  like  library  matching,  except  the  fingerprint  in  (1)  is 
“the  library”  for  each  run 
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BUILDING  STRONG, 


Cluster  Analysis  Basics 


■  Some  Cluster  Analysis  Basics 

►  Need  to  select  a  match  metric 

•  0.95  is  common 

►  Decide  on  a  minimum  number  of  items  that  defines 
“group” 

•  Have  seen  numbers  such  as  3,  4,  5, 10 

►  Only  use  inversion  solutions  that  pass  all  QC 

►  Have  an  expert  on  your  team! 

Let’s  see  how  this  works  using  0.95  match  metric 
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Analyze  For  Clusters  Example 
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Set  this  as  our  “Library” 
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Analyze  For  Clusters  Example 
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Rack  &  Stack’em... 


BUILDING  STRONG, 


Analyze  For  Clusters  Example 


Match  Better  Than  0.95? 


The  basis  of  comparison  signature 
aka  “The  Library” 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


The  Stack 


0.34 


0.01  0.10  1.00  10.00  100.00 


get  B. 

TO. 0000 

1.0000 


Time  (ms) 


— i — i — i  i  i  1 1 1 1 - 1 — i — i  i  1 1 1 1 1 - 1 — i — i  i  i  1 1 1 1 - 1 — i — i  i  1 1 1 1 

x:  438740.74  Phi:  280.5 

y:  3743942.30  Theta29.8 

z:  -0.15  Psi:  -39.4. 


Analyze  For  Clusters  Example 


Match  Better  Than  0.95? 


The  basis  of  comparison  signature 


Time  (ms) 


0.34 


0.01  0.10  1.00  10.00  100.00 


get  B. 

10.0000 

1.0000 


Time  (ms) 

: 1 — i — i  i  i  1 1 1 1 1 — i — i  i  1 1 1 1 1 1 — i — i  i  i  1 1 1 1 1 — i — i  i  ii  i  l 

I  x:  438740.74  Phi:  280.5  ! 

:  y:  3743942.30  Theta29.8  : 

r  z:  -0.1 5  Psi:  -39.4 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


0.01  0.10  1.00  10.00  100.00 
get  C.  Time  (ms) 


1 0  £  . 

=  x:  438741.31 
:  y:  3743933.02 
1  q-i  L  z:  —0.03 


10_H 


cn 
o 

<D 

“  io-H 


i  i  i  i  i  1 1 - 1 - 1 — i  i  i  i  i 

Phi:  163.4 
Thetal  8.9 
Psi:  -7.6 


_ ^ 


0.1000 

0.17 

Analyze  For  Clusters  Example  100 


Match  Better  Than  0.95? 


The  basis  of  comparison  signature 
aka  “The  Library” 

get  A. 
io.< 


Q 


0000 
.0000 
o.^<§$ 
0.0100 

0.0010 

0.0001 


:  x:  438739.55 
:  y:  3743932.28 
_  z:  -0.09 


4- 


Phi:  180.0 
Theta]  .5 
Psi:  16.0  . 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


0.0010  r 


0.0001 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


get  A. 

i  o.oooo  r 


-1 — . I - 1 — . I - ' — . I - 1 — . 


0.95 


1.0000  r 

°o.%c9  \ 

0.0100  r 

o.ooiot 

0.0001 


i  x:  438740.45 
:  y:  3743937.12 
_  z:  -0.07 


Phi:  332.0 
Theta!  1 .0 
Psi:  -34.2. 


0.01  0.10  1.00  10.00  100.00 


0.1000 

0.17 

Analyze  For  Clusters  Example  100 


Match  Better  Than  0.95? 


The  basis  of  comparison  signature 
aka  “The  Library” 

get  A. 
io.< 


Q 


0000 
.0000 
o.^<§$ 
0.0100 

0.0010 

0.0001 


:  x:  438739.55 
:  y:  3743932.28 
_  z:  -0.09 


4- 


Phi:  180.0 
Theta]  .5 
Psi:  16.0  . 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


0.0010  r 


0.0001 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


get  A. 

10.0000 


1.0000 
°oWt 


x:  438740.45 
y:  3743937.12 
.  z:  -0.07 


Phi:  332.0 
Thetal  1 .0 
Psi:  -34.2. 


0.95 


0.0001 


0.01  0.10  1.00  10.00  100.00 


I  .'U'UKJKJ 


a  i  c  n\  c  r 

Analyze  For  Clusters  Example 

0.0100 

Match  Better  Than  0.95?  0.0010 1- 

The  basis  of  comparison  signature 
aka  “The  Library” 

0000 


get  A. 

10. 


.0000  r 


x:  438739.55  Phi:  180.0 

y:  3743932.28  Theta!  .5 

z:  -0.09  Psi:  16.0 


Q 


4- 


0. 

0.0100 

0.0010 
0.0001 

0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


0.0001 


0.01  0.10  1.00  10.00  100.00 


0.11 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


Analyze  For  Clusters  E^nnipl^ 


Match  Better  Than  0.95? 

The  basis  of  comparison  signature 
aka  “The  Library” 


0.1000 

:  \ 

?.0100 

-  *■  \ 

'  \ 

'  A 
Vl 

:  V 

:  r 

0.0010 

0.0001 

. . . 1_ L. 

,  i ' ' 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


get  A. 

10.00001 


1.0000  - 


°oWf 


T 1 1  I  I  I  I  I  | 1 1 1  I  I  I  I  I  | 1 1 1  I  I  I  I  I  | 1 1 1  I  I  I  I 

x:  438739.55  Phi:  180.0 

:  y:  3743932.28  Theta  1 .5 


.  z:  -0.09 


Psi:  16.0  . 


0.0001 


0.97 


10.0000 

E - 1 — . 1 - 1 — . 1 - 1 — . 1 - 1 — . 4 

:  x:  438739.03  Phi:  6.4 

:  y:  3743933.41  Theta8.6 

1 .0000 

r  z:  -0.10  Psi:  -21.5, 

°0  m 

0  \  ‘ 

0.0100 

r  \\  i 

0.0010 

r  \\  i 

0.0001 

:  .  \  •  . : 

0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


0.01  0.10  1.00  10.00  100.00 

rget  B.  Time  (ms) 

1 0.0000  . — . 


1 — i  i  i  1 1 1 1 - 1 — i — i  i  i  i  iy 

l  x:  438738.66  Phi:  6.5 

:  y:  3743933.87  Theta64.2 

1 .0000  r  z:  “O-1  7  psi:  -19-8. 


0.1000 

0.0100 

0.0010 


r\  r\  ^ 


I  .UUUU  r 


Analyze  For  Clusters  Example 

0  97 

0.0100 

Match  Better  Than  0.95?  O.OOIOlr 

The  basis  of  comparison  signature 
aka  “The  Library” 


get  A. 

10.00001 


0.0001 
0.01 

rqet  B. 


-I - 1 - 1  I  I  I  Ml - 1 - 1 - 1  I  I  I  III - 1 - 1 - 1  I  I  I  I  I  I - 1 - 1 - 1  I  I  I  I 


1.0000  r 

°o* 

0.0100 

O.OOIOlr 
0.0001 


;  x:  438739.55 
:  y:  3743932.28 
.  z:  -0.09 


Phi:  180.0 
Thetal  .5 
Psi:  16.0  . 


0.10  1.00  10.00  100.00 
Time  (ms) 


0  0000  i — 1 — 1 — 1 1 1 1 1  m — 1 — 1 — 1 1 1 1 1 1 1 — i — i — 1 1 1 1 1 1 1 — i — i — 1 1 1 1 1 1 


0.13 


0.01  0.10  1.00  10.00  100.00 
Time  (ms) 


1 ,0000  b 
0.1000 

0.0100 

o.ooiob- 

0.0001 
0.01 

'■get  C. 
i  o.oooo  r 


:  x:  438738.66 
:  y:  3743933.87 


.  z:  -0.17 


Phi:  6.5 
Theta64.2 
Psi:  -19.8. 


■  / 

11 


0.10  1.00  10.00  100.00 
Time  (ms) 


And  on. ..and  on. ..and  on... 


1 ,0000  b 
0.1000 
0.0100 
0.0010 


:  x:  438739.50 
:  y:  3743933.59 
.  z:  -0.22 


Phi:  39.3 
Theta48.6 
Psi:  -2.2  . 


Polarizability 


Compare  each  signature  to  all  other 

signatures  on  site 


■  look  for  clusters  (groups  of  items  with  similar  response) 


■  most  clusters  correspond  to  things  we  know  about,  like 
these  two  examples 


_Q 

CD 

N 

_cd 

o 

CL 


100 

10 

1 

0.1 

0.01 

0.001 


Time  (ms) 


BUILDING  STRONG, 


Willi  I  I  I - |l  1 1 1 1  I  I  I  |l  1 1 1 1  I  I  I  gill  1 1  I  I  I  |l  1 1 1 1  I 


Compare  each  signature  to  all  other 

signatures  on  site 


►  If  there  are  clusters  of  items  that  do  not  match  the  library 
signatures,  we  excavate  some  of  them  and  proceed 
accordingly... 


0.1  1  10 
Time  (ms) 


T-Bar  Fuze 

Non-hazardous  clutter, 
did  not  add  to  library 


BUILDING  STRONG, 


Classification  Techniques 

Summary 


■  Library  Matching 

►  Asks  what  an  unknown  target  “looks  like”  in  EMI  sense 

►  Compares  polarizability  against  bank  of  signatures  for  expected 
munitions  and  other  training  objects 

■  Be  a  Cluster  of  things  of  interest 

►  Same-shaped  objects  have  the  same  polarizabilities 

►  Groups  of  identical  things  that  turn  out  to  be  UXO 

■  Be  Large,  long,  axisymmetric  and  thick-walled 


How  it  works 
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BUILDING  STRONG, 


Something  New... sort  of... 


■  Informed  Source  Selection  (formerly  called  Informed 

Source  Identification  and  Selection  until  Abu  Bakr  al-Baghdadi  was 
named  as  Caliph!) 


How  it  works 

■  Uses  new  sensors  in  dynamic  (detection)  mode 

■  Collects  lower  volume  of  data,  at  a  much  higher  rate 
(~10Hz  vs.  0.01 -0.03Hz) 

■  Uses  same  inversion  process  to  get  size  and  wall 
thickness 

■  Culls  really  small  anomalies  off  the  cue  list 


72 


BUILDING  STRONG, 


Two  Stage  Process 


■  Stage  1 :  Find  anomaly 
locations 


Ch:  t_1 0_filterdist  -  Center  5  Z  (MN  2  3  4  5  6) 


114  114.5  115  115.5  116 

Easting  -  531800  m 


§  142.5 


■  Stage  2:  Calculate 
parameters  and  screen 


Stage  1 :  Detect  Anomalies 

Two  ways  of  doing  this: 


Look  For  Peaks  Look  For  Metal 


■  TEMTADS  TzRz  Signal  ■  Dipole  Inversion  Output 


JA 


BUILDING  STRONG, 


Along -track  position  (m) 


Detect  Anomalies:  Look  For  Peaks 

WAIT!  What  Threshold  Do  We  Use? 


■  Three-steps  process: 

■  Step  1 :  Calculate  threshold  for  vertical  (z)  component  data 

►  All  calculations  for  smallest  or  most  difficult  TOI  to  detect 

►  All  calculations  for  worst  case  scenario 

►  Artificially  add  noise  &  superimpose  on  real  site  data 


BUILDING  STRONG, 


Detect  Anomalies:  Look  For  Peaks 

WAIT!  What  Threshold  Do  We  Use? 


Step  2:  Synthetic  Seeding  &  Monte  Carlo 
Simulations 


Ch:  t  10  filterdist  -  Center  5  Z  (MN  2  3  4  5  6) 

•  tfe  ♦  a*  t  mm  v* 

*  '  *  w  *  (fft 

4’*’  '  ‘  ■  t*  *  ’£■ 

.  *  'V 

*  A  M  * 

V*  •  •  :  ♦  „  '*  ♦ 

^ .  <?.  .  *»/  ,  • 

Can  Do  This  For 

Any  Number  Of 

to  .Hr 

TOI 

i’  ••  ♦ 

(shown  here: 

*’  ;  *'*m 

20mm  TP  added 

t 


V  % 


at  white  triangle 
locations) 


BUILDING  STRONG, 


Decay 


Step  3:  Invert  &  Analyze  All  Monte  Carlo  Simulations 

■  Run  regular  inversion  at  each  detected  anomaly  (figure  below) 

■  Adopt  a  conservative  approach:  only  anomalies  that  can  be 
declared  non-TOI  with  high  confidence  can  be  removed 


-2-101234 

Size 


Example  for  3  small 
artificially  seeded  TOI: 

20mm  TP  (orange  circles) 
20mm  AP  (yellow  circles) 
37mm  AP  (grey  stars) 


->  all  other  detected 
anomalies:  Purple  dots 

BUILDING  STRONG, 


Detect  Anomalies:  Look  For  Metal 


■  Invert  a  square  meter  of  data  every  1 0cm  assuming 
a  smallest  TOI  or  larger  piece  of  metal  is  present 

■  Resulting  Map  =  How  well  inversion  matches  data 


Peak  Detection  Map 


Inversion  Match  Metric  Map 

_ BSB 


CiXiDS 


za 


BUILDING  STRONG, 


Where  We  Are  Now 


Where  are  today 


79 


BUILDING  STRONG, 


Diversity  of  Deployment  Platforms 


Ellis 


►  MetalMapper 


BUILDING  STRONG, 


Diversity  of  Deployment  Platforms 


►  TEMTADS  2x2 


BUILDING  STRONG, 


Diversity  of  Deployment  Platforms 


♦  MPV 


Dynamic  (GPS  or  Line/Fiducial) 


Cued  (local  Beacon  positioning) 


BUILDING  STRONG, 


Potential  Savings  on  a  1 00-acre  Site 


Mob/Demob 

1 

Surface  Sweep 

acre 

IVS 

each 

Seed  Emplacement 

per  day 

EM61  Data  Collection  and  Analysis 

acre 

Dynamic  TEMTADS  Collection  and  Analysis 

acre 

Cued  TEMTADS  Collection  and  Analysis 

per  anomaly 

Intrusive  Investigation 

per  dig 

$25,000  range:  $15,000  to  $30,000 

$1 ,500  range:  $500  to  $5,000 

$6,000  One  day  of  three-person  crew 
$5,250  assumes  25  seeds  emplaced  per  day,  crew  size  of  3 
$1 ,000  range:  $1 ,000  with  array  to  $5,000  with  single  sensor 
$6,000  range:  $3,300  to  $5,500 
$40 

$120  range:  $75  to  $200 


Traditional 
Approach  -  No 
Classification 

Mob/Demob 

=  unit  costs 

$25,000 

Surface  Sweep 

=  1 00  acres  *  per  acre 

$150,000 

IVS 

=  unit  cost 

$6,000 

Seed  Emplacement 

=  25  QC  +  25  QA 

$10,500 

EM61  survey  and  analysis 

=  100  acres  *  DGM  costs 

$100,000 

Dynamic  TEMTADS 

n/a 

Cued  TEMTADS 

n/a 

Seeds  Dug 

=  seeds  *  cost  per  dig 

$6,000 

Native  UXO  Dug 

=  #  UXO  *  cost  per  dig 

$60,000 

Clutter  Dug 

=  #  clutter  *  cost  per  dig 

$5,940,000 

Fixed  Costs 

$400,000 

Total 

$6,697, 

500 

Classification 

same  as  traditional 

$25,000 

same  as  traditional 

$150,000 

same  as  traditional 

$6,000 

=  200  QC  +  200  validation 

$84,000 

n/a 

=  100  acres  *  TEMTADS  costs 

$600,000 

=  50%  reduction  from  advanced  analysis 

$1,000,000 

=  seeds  *  cost  per  dig 

$48,000 

same  as  traditional 

$60,000 

=  80%  clutter  rejection 

$1,188,000 

$3,561 

$400,000 

,000 

BUILDING  STRONG** 

F*1 

Not  Only  Direct  Cost  Savings 


■  Fewer  holes  ->  less  environmental 
damage 

►  Wetlands 

►  Sensitive  environments 

►  Golf  courses 

■  Fewer  digs  ->  less  disruption  and  fewer 
evacuations 


BUILDING  STRONG, 


Is  Classification  Applicable  at  This  Site? 


■  If  EM61  or  Magnetometer  Today,  Then  YES! 

■  What  are  the  targets  of  interest  at  this  site? 

►  Historical  research 

►  Recovered  munitions  and  fragments/scrap 

►  Depth  and  density 


Workflow  and  Classification  Planning 


BUILDING  STRONG, 


Classification  Planning 


■  What  classification  sensor  is  appropriate  for  the 
site? 


►  Benign  terrain 

•  MetalMapper 

►  Rougher  terrain  with  steeper  slopes  and  trees 

•  TEMTADS  2x2.  MPV.  Handheld  BUD 


MetalMapper 


TEMTADS  2x2 


MPV 


Photos  courtesy  ESTCP 

Workflow  and  Classification  Planning 


BUILDING  STRONG, 


Classification  Planning  (cont.) 


What  parts  of  the  site  have  an  anomaly  density  that  will  allow 
classification  to  be  successful  and  at  what  cost? 

►  Small  sites  and  densities  greater  than  3K  to  4K  per  acre 
may  not  be  appropriate 

►  For  sites  with  few  anomalies,  the  costs  of  remediation 

must  justify  the  extra  expense  required  for  data  collection 

•  Wetlands,  chemical  sites,  sensitive  habitat,  disruptive  evacuations, 
land  owner  needs,  public  interest  etc. 

Not  Appropriate  .  . 

.  .  ■  >  Appropriate 


l 


Landfill  Surprise 


40-mm  Anti-Aircraft 
Projectile 

Photos  courtesy  U.S.  Navy 


BUILDING  STRONG, 


Classification  -  Where  We  Stand  And  What  It 

Means 


Density  From  Dynamic  TEMTADS 


Target  Count  Estimates: 

Inside  High  Density  area:  3,913 

Outside  High  Density  Area:  14.925 

Total:  18,838 


High  Density  Area 
( >4,000  targets/acre) 


4743.6 

4487.2 

4230.8 

3974.4 

3717.9 

3461.5 

3205.1 

2948.7 

2692.3 

2435.9 

2179.5 

1923.1 

1666  7 

1410.3 

1153.8 

897.4 

641.0 

384  6 

126  2 

Density 

(targets/acre} 

Former  Camp  San  Luis  Obispo 
37  mm  projectiles  to  155mm  projectiles 


It  works  a  lot 
better  than  we 
imagined 

►  Even  with 
seven  different 
UXO  in  the  site 

►  Even  with 
37mm 
projectiles 

►  Even  in  up  to 
4,000 

anomalies  per 
acre 


BUILDING  STRONG, 


Example 

Full-Scale  Application 
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BUILDING  STRONG, 


Example  Site 


University  owned 

►  No  access  restrictions 

►  Cattle  grazing 

►  Geotechnical  classes 

►  Camping 

Multiple,  overlapping 
range  fans 

MRS  ~  2,500  acres 

100  acres  in  Year  1 


BUILDING  STRONG, 


Site  History 


■  Initially  established  in  1928 

■  Expanded  during  WWII 

■  Transferred  to  private  owners  after  Korean  War 

■  Previous  investigations 

►  Preliminary  Assessment  (1986  and  1993) 

►  Time  Critical  Removal  Action  (1992) 

►  Archives  Search  Report  (1994) 

►  Site  Inspection  (2007) 

►  Time  Critical  Removal  Action  (2010) 

►  Remedial  Investigation  (2011) 


BUILDING  STRONG, 


CSM  -  Spatial  Distribution  of  MEC 


BUILDING  STRONG, 


Depth  (meters  below  ground  surface) 


CSM  -  Vertical  Distribution  of  MEC 


deepest 
recovered  UXO 


amplitude  response 
detection  limit 


bedrock  in 
valleys 


BUILDING  STRONG, 


Remedial  Objective 


■  Detect  and  dispose  of  MEC  that  can  be 
detected  using  a  detection  threshold  required 
to  detect  a  37mm  projectile  at  12  inches 
below  the  ground  surface,  and  to  do  so  as 
efficiently  as  possible 

►  Remove  any  MEC  detected  irrespective  of  depth 

►  As  efficiently  as  possible  =  most  economical 
method  to  accomplish  remedial  objectives 


BUILDING  STRONG, 


Projected  Costs  for  Year  1 


Mob/Demob 

1 

$25,000 

range:  $15,000  to  $30,000 

Surface  Sweep 

acre 

$1,500 

range:  $500  to  $5,000 

IVS 

each 

$6,000 

One  day  of  three-person  crew 

Seed  Emplacement 

per  day 

$5,250 

assumes  25  seeds  emplaced  per  day,  crew  size  of  3 

EM61  Data  Collection  and  Analysis 

acre 

$1,000 

range:  $1 ,000  with  array  to  $5,000  with  single  sensor 

Dynamic  TEMTADS  Collection  and  Analysis 

acre 

$6,000 

range:  $3,300  to  $5,500 

Cued  TEMTADS  Collection  and  Analysis 

per  anomaly 

$40 

Intrusive  Investigation 

per  dig 

$120 

range:  $75  to  $200 

Traditional  Approach  -  No  Classification 

Classification 

Mob/Demob 

=  unit  costs 

$25,000 

same  as  traditional 

$25,000 

Surface  Sweep 

=  1 00  acres  * 

per  acre 

$150,000 

same  as  traditional 

$150,000 

IVS 

=  unit  cost 

$6,000 

same  as  traditional 

$6,000 

Seed  Emplacement 

=  25  QC  +  25  QA 

$10,500 

=  200  QC  +  200  validation 

$84,000 

EM61  survey  and  analysis 

=  1 00  acres  * 

DGM  costs 

$100,000 

n/a 

Dynamic  TEMTADS 

n/a 

=  100  acres  *  TEMTADS  costs 

$600,000 

Cued  TEMTADS 

n/a 

=  50%  reduction  from  advanced  analysis 

$1,000,000 

Seeds  Dug 

=  seeds  *  cost  per  dig 

$6,000 

=  seeds  *  cost  per  dig 

$48,000 

Native  UXO  Dug 

=  #  UXO  *  cost  per  dig 

$60,000 

same  as  traditional 

$60,000 

Clutter  Dug 

=  #  clutter  *  cost  per  dig 

$5,940,000 

=  80%  clutter  rejection 

$1,188,000 

Fixed  Costs 

$400,000 

$400,000 

Total 

$6,697,500  |  | 

$3,561 ,000 

BUILDING  STRONG, 


Intial  TPP  Meeting 


■  Agree  on: 


►  Remedial  Objective 

►  Survey  lane  spacing 

►  Anomaly  selection  methodology 


•  Advanced  detection  methods 

►  Sensor  Selection 

►  Schedule 

►  . 


MR  Project  Work  Elements  that  Classification 

Significantly  Changes 


■  GIS  setup 

■  Document  management 
and  control 

■  Subcontracting 

■  Technical  and 
operational  approach 

■  Work  Plan  preparation 
and  approval 

■  Site  prep  and 
mobilization 

■  Site  survey/grid  layout 

■  Vegetation  removal 

■  Surface  removal 


Geophysical  System 

Verification  (GSV) 

Geophysical  survey,  data 

collection,  and  processing 

Anomaly  reacquisition  and 

investigation 

MEC/MPPEH  management 

Demobilization 
Final  report 
Archiving 
Project  closeout 


BUILDING  STRONG, 


Geophysical  Classification  for  Munitions 
Response-Quality  Assurance  Project  Plan 

(...or  GCMR-QAPP) 


INTERGOVERNMENTAL  DATA  QUALITY  TASK  FORCE 


■  What  follows  is 
based  on  beta  draft 
template 


Uniform  Federal  Policy  for  Quality 
Assurance  Project  Plans  Template 

Geophysical  Classification  for  Munitions  Response 


Revised  Beta  Draft 
12/30/2014 


BUILDING  STRONG, 


Project  Workflow 


TPP  — ►  QAPP 


Site  Prep 


Cued  Survey 


Intrusive 

Investigation 


Verification  and 
Validation 


DFWs 


1,2 


3,4,5 


6,7 


9,10 


11 


12 


Example  Tasks 

Surface  Clearance 
QC  &  QA  Seed  Emplacement 
IVS  Construction 

Dynamic  Survey 
Anomaly  Selection 

Background  Data  Collection 
Anomaly  Data  Collection 

"Training"  Digs 

Construct  Ranked  Anomaly  List 


Dig  Items 

Compare  Recoveries  to  Predictions 


Verify  Thresholds 
Validate  Process 


Products 

Surface  Clearance  Tech  Memo 
QC  Seed  Emplacement  Memo 
Validation  Seed  Memo  (Govt  Only) 

IVS  Tech  Memo 
Weekly  QC  Submittal 
Anomaly  Selection  Memo 

IVS  Tech  Memo 
Weekly  QC  Submittal 

Classification  Tech  Memo 
Ranked  Anomaly  List 

Intrusive  Investigation  Report 
Photos 

Final  Draft  Validation  Plan 

Final  Report 


BUILDING  STRONG, 


Surface  Clearance  Memo 


■  Was  everything  found  consistent  with  the  CSM? 

■  Is  there  anything  the  analysts  need  to  know? 

■  Opportunity  for  cost  savings 


37-mm  projectile 

Y 

2.36-in  rocket 

Y 

60-mm  mortar 

Y 

75-mm  projectile 

Y 

81 -mm  mortar 

N 

3-in  stokes  mortar 

Y 

unexpected  munition 


N 


BUILDING  STRONG 


Contractor’s  QC  Seeds 


■  Inert  37-mm  projectiles  and  small 
ISO80s 

■  200  QC  seeds,  approximately  50%  of 
each 

■  Seeds  placed  at  six  depths  up  to  the 
maximum  PWS  detection  depth  (5, 
10,  15,  20,  25,  &30  cm) 


Government  Validation  Seeds 


TPP  QAPP 

1 

■  -200  seeds 

Site  Prep 

■  Small  ISO80,  37-mm 

1 

Detection 

Survey 

projectiles,  mortars 
■  Full  depth  range  of  interest 

1 

Cued  Survey 

I 

1  ►  30  cm  for  small  ISO  and 

Classify 

Anomalies 

37mm 

►  45  cm  for  60-mm  mortar 

1 

Intrusive 

Investigation 

1 

►  65cm  for  81 -mm  mortar 

Verification  anc 
Validation 

1 

Rllll  niNfi  STRONG  E23 

IVS  Construction 


BUILDING  STRONG, 


IVS  Tech  Memo 


Is  the  Sensor  Functioning  Properly 
and  Ready  to  Collect  Data? 


BUILDING  STRONG, 


IVS  Tech  Memo 


Are  the  Remedial  Objectives  Achievable? 


TEMTADS  2x2  Sensor  Response  Curves  for 
2  Variants  of  37-mm  Projectile 


depth  bgs  (cm) 


BUILDING  STRONG, 


Detection  Survey 
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Monostatic  Z  Component  Response  Amplitude 


MonoZ_5_lev 

(mV/A) 


BUILDING  STRONG, 


Anomaly  Selection  Methodology 


■  Amplitude  Threshold 

►  Small,  near-surface  frag  results  in  amplitude 
equal  to  deeper  targets  of  interest  -  lots  of 
unnecessary  “detections” 

■  Informed  Source  Selection 

►  Use  all  channels  of  data  collected  from  all 
receivers  in  analysis.  Only  flag  anomalies  that 
result  from  items  big  enough  to  be  the  smallest 
TOI  (37-mm  projectile  in  this  case). 


BUILDING  STRONG, 


Anomaly  Selection 


Advanced  Anomaly  Selection 


Reduced  detections  by  50%  on  average 

 ^.DING  STRONG^ 


Detection  of  QC  Seeds 


BUILDING  STRONG, 


Cued  Data 


f 

©on 

►  Libraries  ►  Documents  ►  TEM  Data  Files  ►  14August2014 

t  |  H  Search  14A...  p  | 

Organize  »  Share  with  »  New  folder  |=z  ▼  ^ 


Documents 

14August2014 


Name  Date  modified  Type  Size 


0  SLO Yea  rl G  ri  d46 SB  G 00G0G1 2014028 00 .  h5 

8A4/2014  8:13  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl G  ri  d46 SAM 001297 2Q14G23 QQ .  h5 

8/14/2014  8  :18  AM 

NCSA  HDFView 

35  KB 

@  SLO  Yea  rl_G  ri  d46_SAM_001298_2G14028_00 .  h5 

8/14/2014  8:21  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_G01299_2014023_G0 .  h5 

8/14/2014  8:25  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001300_2014023_00 .  h5 

8/14/2014  8:28  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_G01301_2G14G28_G0 .  h5 

8/14/2014  8:31  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001302_2014028_00 .  h5 

8/14/2014  8:34  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001303_2014028_00 .  h5 

8A4/2Q14  8:38  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001304_2014028_00 .  h5 

8/14/2014  8:42  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001305_2014028_00 .  h5 

8/14/2014  8:45  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001306_2014028_00 .  h5 

8/14/2014  8:48  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001307_2014028_00 .  h5 

8/14/2014  8:51  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001308_2014028_00 .  h5 

8/14/2014  8:55  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_G013G9_2014023_GG .  h5 

8A4/2Q14  8:58  AM 

NCSA  HDFView 

35  KB 

@  SLO  Yea  rl_G  ri  d46_SAM_Q01310_2Q14Q2S_GQ .  h5 

8A4/2014  9:02  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001311_2014023_GG .  h5 

8A4/2014  9:05  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001312_2014023_GG .  h5 

8A4/2014  9:08  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl G  ri  d46 SAM 001313 2Q14028 QQ .  h5 

8A4/2014  9:12  AM 

NCSA  HDFView 

@  SLO  Yea  rl G  ri  d46 SB  G G00GG2 2G14G28 GG .  h5 

0  SLO  Yea  rl_G  ri  d46_SAM_001314_2014Q28_00 .  h5 

8A4/2014  9:22  AM 

NCSA  HDFView 

0  SLO  Yea  rl_G  ri  d46_SAM_001315_2014028_00 .  h5 

8A4/2014  9:25  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001316_2014Q28_00 .  h5 

8A4/2014  9:28  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001317_2014028_00 .  h5 

8A4/2014  9:32  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001318_2014028_00 .  h5 

8A4/2014  9:35  AM 

NCSA  HDFView 

35  KB 

0  SLO  Yea  rl_G  ri  d46_SAM_001319_201402S_0Q .  h5 

8A4/2014  9:38  AM 

NCSA  HDFView 

35  KB 

27  items 
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PUILUINVa  °  1 


Government  Evaluation  of  Validation 
Seed  Locations  after  Inversions 


0 


270 


90 


180 


BUILDING  STRONG, 


Identify  Clusters 


Cluster  #2  Cluster  #3 

(18  matches)  (41  matches) 


Cluster  Match  Threshold  =  0.95 


BUILDING  STRONG, 


I  Mill  I  I  I  I  Mill  I  I  I  I  Mill  I  I  I  I  Mill 


Intrusive  Investigation 


Cumulative  Anomaly 


BUILDING  STRONG^ 


Intrusive  Investigation  Report 


Were  all  recovered  items  consistent  with  analyst's  predictions? 


1.00 


1  095 

V-i 

TO 

CO 

c 

o 

'(/) 

®  0.90 


0.85 

0  500  1000  1500  2000  2500 


t — ] — i — i — i — i — [ — i — i — i — i — ] — i — i — i — i — | — i — p — i — ! — | — i — i — r 

momaly  46_00056 

Anomaly  76  01015 

«  J  —  Clutter 

\  *  T0! 


Anomaly  22_00327 


★  uxo 


\ 


’  •  - 


Dig  Number 


BUILDING  STRONG^ 


Intrusive  Investigation  Report 


Anomaly  46_00056 
Decision  Metric  =  0.9896 


BUILDING  STRONG, 


Intrusive  Investigation  Report 


Anomaly  76_01015 


0.1  1  10 

Time  (ms) 


Anomaly  22_00327 
Decision  Metric  =  0.9855 


Verification  and  Validation 


Verify  the  stop-dig  threshold 

►  dig  past  the  last  TOI 


BUILDING  STRONG, 


Contractor  Stop  Dig  Point 


TPP  QAPP 


I 


BUILDING  STRONG, 


Verify  The  Threshold 


TPP  QAPP 


I 


BUILDING  STRONG, 


Verification  and  Validation 


TPP  QAPP 

1 

■  Verify  the  stop-dig  threshold 

Site  Prep 

1 

►  dig  past  the  last  TOI 

Detection 

Survey 

■  Validate  the  whole  process 

1 

Cued  Survey 

►targeted  investigation  of  items 
classified  as  likely  clutter 

I 

Classify 

Anomalies 

1 

Intrusive 

Investigation 

1 

Verification  anc 
Validation 

1 

Rllll  niNfi  STRONG  E23 

Validate  the  Process 


TPP  QAPP 


I 


BUILDING  STRONG, 


After  Action  Vertical  CSM 


I  I  seed  interval 

-  uxo 

-  inert 

★  j  ★  deepest  recovery 
—  —  —  detection  depth 


BUILDING  STRONG, 


Everybody  Is  Happy  Now! 


BUILDING  STRONG, 


Summary 


BUILDING  STRONG 


Classification  Basics  -  Summary 


■  Modern  sensors  Reliable  classification 

■  Analysis  procedures  Extract  intrinsic  EMI 
signature 

►  Depends  only  on  size,  shape  and  material  properties  of 
target 

■  Library-based  classifiers-^ Reliably  distinguish 
between  munitions  and  clutter  items 

►  Match  unknown  targets  with  other  objects  with  similar  EMI 
signatures  (i.e.,  things  they  “look  like”) 

►  Presumptive  UXO  (dig)  if  EMI  “vision”  is  fuzzy  or  obscured 


BUILDING  STRONG, 


Where  Do  We  Stand  Today? 


VCommercially-available  sensor 

V  Available  analysis  tools 
VTrained  contractor  base 
VTrained  Government  geophysicists 

V  Regulator  and  Stakeholder  acceptance 

^Government  business  practices 
^Government  PM  acceptance 


BUILDING  STRONG, 


On-Going... 


Transfer  of  the  technology  to  mainstream  use 
Some  of  the  challenges 

ifety  policies 
Mrty - 

la-nguage  and  request  for  proposal 


►  BoD 

►  eq 

►  contr 
requir 

►-quality  contr 


assurance  processes 


►  workforce/practitioner  qualifications 


BUILDING  STRONG, 


Additional  Training  and  Resources 

■  SERDP/ESTCP  website  contains  webcasts,  tutorials, 
and  demonstration  reports  on  the  classification 
technology 

►  www.serdp-estcp.org 

■  ESTCP  Technology  Transfer  Workshops 
(June  6-7  201 2  &  June  1 0-1 1  201 5  Golden,  Colorado) 

►  Results  of  site  demonstrations,  outdoor 
technology  demonstrations,  tutorials,  and  software 

■  M2S2  Webinar  Series 

►  http://www.clu-in.org/conf/tio/m2s2fv1 5-2  02261 5/ 

►  http://www.clu-in.org/conf/tio/m2s2fv1 4-3  02251 4/ 

►  http://www.clu-in.org/conf/tio/m2s2-3  04221 3/ 

■  Interstate  Technology  Regulator  Council 

Geophysical  Classification  For  Munitions  Response:  http://www.itrcweb.org/gcmr-2/ 
Archived  web  training:  https://clu-in.org/conf/itrc/GCMR  01 1 41 6/ 
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BUILDING  STRONG, 


Army  UXO  Safety  Program 


Legit)  gn4  Follow 

the  3R$ 

RECOGNIZE:  The  danger  lhai  a  souvenir 
munition  poses  to  yourself, 
your  family  and  your  neighbors 

RETREAT;  Do  not  disturb,  touch  or  move  it 
Do  nor  give  or  throw  it  owcy 

REPORT:  Coll  911 


BUILDING  STRONG, 


THE  VERTICAL  CONCEPTUAL 
SITE  MODEL,  AND 
VISUAL  SAMPLE  PLAN 


Advanced  Geophysical 
Classification  (AGC) 


US  Army  Corps 
of  Engineers. 


r 


CONCEPTUAL  SITE  MODELS 

IN  THE  BEGINNING... 


L 


US  Army  Corps 
of  Engineers, 


THE  MMRP 

Munitions  Response 

Process 


Removal  Response 


Removal 

Needed 


months  /Nq* 


Remedial  response 


/m 


NTCRA 

*  EE/CA 

*  AM 


'MECWIC\ 

confirmed 

t  oatentlat  j 


Start  Here! 


End  Here! 


Proposed 


— 


Decision 


Remedial 

Action 


Remedial 

Design 


LTM 


Public 
Review 
30  days 


r 


PRELIMINARY  ASSESSMENT:  FURTHER  RESPONSE 
NEEDED  -»THE  INITIAL  CSM 


Areas  suspected  to 
contain  MEC... 
Now  What? 


L 


US  Army  Corps 
of  Engineers 


SI:  PRESENCE  OR  ABSENCE? 


L. 


US  Army  Corps 
of  Engineers 


REMOVAL  ACTIONS  -  MIGHT  BE  NEEDED  SOMETIMES 


/ 


Maybe  a 
TCRA? 


US  Army  Corps 
of  Engineers 


THE  MMRP 

Munitions  Response 

Process 


Removal  Response 


Removal 

Needed 


months 


Remedial  response 


/m 


NTCRA 

*  EE/CA 

*  AM 


'MECWIC\ 
confirmed 
^  uotentialv 


Proposed 


— 


Closeout 


You  are  here 


Decision 


Remedial 

Action 


Remedial 

Design 


Public 
Review 
30  days 


PURPOSE  FOR  THE  Rl 

To  investigate  the  project  area  to  define  nature  and 
extent  of  release,  and  determination  of 
“acceptable  versus  unacceptable  risk  to  support 
a  recommendation  for  one  of  the  following: 

-  No  action  is  appropriate  (Acceptable/Protective) 

-  Conduct  Feasibility  Study  (Unacceptable/not 
Protective) 

-  Collect  additional  data  via  an  “expanded”  Rl 
(Inconclusive  Protectiveness  /  Characterization) 


US  Army  Corps 
of  Engineers 


EPA  RI/FS  GUIDANCE 


9 


“The  objective  of  the  RI/FS  process  is  not  the 
unobtainable  goal  of  removing  all  uncertainty,  but 
rather  to  gather  information  sufficient  to  support 
an  informed  risk  management  decision  regarding 
which  remedy  appears  to  be  most  appropriate  for 
a  given  site.” 

1  Guidance  for  Conducting  Remedial  Investigations  and  Feasibility  Studies  under 
CERCLA,  U.S.  EPA,  October  1988 

Note  that  the  Remedial  Investigation  and  Feasibility  Study  (RI/FS)  share  the  same 
objective. 


US  Army  Corps 
of  Engineers 


r  ^ 

UPDATED  CSM 

The  post-RI  CSM  forms  the  basis  for  the  Remedial  Design, 
to  include  Advanced  Geophysical  Classification. 

-  What  is  the  unacceptable  risk? 

-  What  will  we  look  for? 

-  Where  will  we  look? 

-  To  what  depth? 

-  What  is  the  Remedial  Action  Objective? 


L 


US  Army  Corps 
of  Engineers 


r  ^ 

BASELINE  RISK  ASSESSMENT 

The  baseline  risk  assessment  should  demonstrate  whether 
there  is  unacceptable  risk  associated  with  DoD  releases. 

-  Toxicity  and  concentration  of  contaminants 

-  Fate  and  transport  of  contaminants 

-  Current  and  potential  future  land  use 

-  Current  and  potential  receptors  (human  and  ecological) 


L 


US  Army  Corps 
of  Engineers 


12 


CONCEPTUAL  SITE  MODELS 


L. 


US  Army  Corps 
of  Engineers 


r  'l 

CONCEPTUAL  SITE  MODEL  (CSM) 

Basic  understanding  of  what  is  going  on. 

-  Where  MEC  is  expected 

-  Identifies  receptors 

-  Describes  exposure  scenarios  (land  use) 

Forms  basis  for  communication  with  stakeholders. 

Assists  in  developing  investigation  strategy  and  DQOs. 

A  CSM  is  a  model  that  illustrates  what  we  know  and 
suspect  is  going  on  at  an  MRS. 
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of  Engineers 


Let’s  look  at  some  different  types  of  models 
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QUIZ! 


Of  the  seven  preceding 
models,  which  TWO  have 
been  used  in  FUDS 
MMRP  reports? 
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INITIAL  CSM 


Tell  the  Story 


There  once  was  a  camp.... 


US  Army  Corps 
of  Engineers 


They  REALLY  used  it.... blew  some  stuff  up. 


US  Army  Corps 
of  Engineers 


and  they  practiced  and  practiced. 


LAND  USE:  WHY  WE  NEED  TO  KNOW 


LAND  USE 


LAND  USE 


- —  «“ "  “  ™ 


O  O  O  lO  o 
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MMRP  &  HTRW  DIFFERENCE  IN  CSM  CONCEPTS 


33 


HTRW:  source  ->  receptor 
MMRP:  source  <-  receptor 
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POTENTIAL  Rl  RESULTS 


US  Army  Corps 
of  Engineers 


Ail  Other  Areas 
(MRS  3) 


Suspected  Target  Area  #1 
(MRS  1) 


Suspected  Target  Area  #2 
(MRS  2) 


This  example  shows  209  grids  (50’  x  50’)  in  MRS  3. 


US  Army  Corps 
of  Engineers 


r  ^ 

VERTICAL  CSM 

Critical  for  planning  the  remedial  action. 

-  What’s  out  there? 

-  How  deep  is  it? 

-  Can  we  reliably  detect  it,  and  with  what  degree  of 
certainty? 
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75M^I 

•  Fuzes/Boosters 
o  MD 

•  20MM 

•  Low  Ordep  HE 

•  Partial  HE  Projectile 
PD  Element 


FIRST  CUTS  AT  [X]  VS.  DEPTH 


Where  are  the  munitions? 

0  50  100  150  200  250  300  350  400  450 


CHARACTERIZATION:  VERTICAL 

DISTRIBUTION? 
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HOW  DOES  ALL  THIS  TIE-IN  TO  AGO? 


L. 
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AGC  &  Visual  Sample  Plan  in  the 
Remedial  Investigation 


US  Army  Corps 
of  Engineers 


THE  FUNDAMENTALS  DO  NOT  CHANGE 
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Study  Boundary 


VSP  Analysis  (Kriging)  using  TEMTADS 

transects  (assuming  0.8m  footprint)  and  Krijsfd  EillmBlCJ 

anomaly  selections.  ( Anomalic*  Acre) 


Treatability  Study  Site 
Boundary 


TEMTADS  Transects 
and  Selected 
Anomalies 


ii 

t 


Kriged  Anomaly  Density 


y 


TfVrr 


Anomalies  /  Acre 


Visual  Sample  Plan 

•  Find  Target  Areas 
(areas  of  Concentrated 
Munitions  Use) 

•  Estimate  upper  bound 
of  potentially  remaining 
UXO  (buffer  areas  or 
unused  lands) 

•  Estimate  numbers  of 
anomalies 
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EXAMPLES  OF  CMUA  AND  ANOMALY  ANALYSES 


OSLO 

IGE  Anomaly  Estimate: 
15,000 

Actual:  19,000 
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IGE  Anomaly 

Estimate: 

-7,000 
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AGC  BENEFITS  IN  THE  Rl 


Conventional  Approach 

Transects:  $200K 
Grids:  $200K 
Cueing:  $0 
Digging:  $400K 


Project  Total:  $1 M 

Total  #  Anomalies  with 
answers:  2,000  to  3,000 
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AGC 

Transects:  $200K 
Grids:  $200K 
Cueing:  $300K 
Digging:  $100K 


Project  Total:  $1M 


Total  #  Anomalies  with 
answers:  -8,500! 


US  Army  Corps 
of  Engineers 


Uniform  Federal  Policy 

for  Quality  Assurance  Project  Plans 

(UFP-QAPP) 


AGC  Training  QAPP  Introduction 


Movie:  Manager's  Role  in 
Assuring  Data  Quality:  Overview 
of  the  Uniform  Federal  Policy  for 
Quality  Assurance  Project  Plans 


View/download: 

https://www.epa.gov/sites/production/files/fedfac/manaqer_role_qualitv_data.mpq 


Historical  Overview 


■  Inspectors  General 

■  Cited  data  quality  issues 

■  Task  force  formed 
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Uniform  Federal  Policy 

Intergovernmental  Data  Quality  Task  Force 
(IDQTF) 


■  Consensus  workgroup 

■  Representatives  from  EPA  Headquarters 
and  Regions,  DoD  services,  and  DOE 

■  Navy  chair  DoD  Environmental  Data  Quality 


Workgroup 

■  NAVSEA  Laboratory  Quality  and  Accreditation  Office 
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Information  Quality  Guidelines 


■  Required  by  the  Data  Quality  Act-  PL106-554 
(2001) 

■  ...for  Ensuring  and  Maximizing  the  Quality, 
Objectivity,  Utility  and  Integrity  of  Information... 

■  In  essence,  requires  that  data  have  a  pedigree 
that  permits  users  and  potential  users  to  assess 
usability  of  data  for  specific  purposes 

■  A  good,  implemented  QAPP  is  the  foundation  of 
data  pedigree 


UFP  for  Implementing  Quality 
Systems 


First  Product  of  I DQTF 

.  Signed  by  EPA  and  DOE  (2002) 
■  Signed  by  DoD  (2003) 


■  Based  on  Quality  Systems  for 
Environmental  Data  Collection  and 
Environmental  Technology  Programs, 
ANSI/ASQ  E4  Section  5  (Part  A) 

■  Section  5,  Management  Elements  of  a 
Quality  System 


UFP  for  Quality  Assurance 
Project  Plans  (UFP  QAPP) 

Based  on  ANSI/ASQ  E4  Section  6  (Part  B) 


■  Project-Specific 

■  Consistent  with  EPA  QA/R-5  and  QA/G-5 


■  Organized  around  four  major  QAPP 
elements 

■  Project  Management  and  Objectives 

■  Measurements  and  Data  Acquisition 

■  Assessment  and  Oversight 

■  Data  Review 


UFP-QAPP  Implementation 


■  UFP-QAPP  is  voluntary  consensus  policy 

■  Once  adopted  by  Federal  department,  agency,  or 
program,  use  is  mandatory  within  that 
organization 

■  UFP-QAPP  Manual  signed  by  EPA  (2004),  DoD 
(2005) 

■  OSWER  Directive  9272.0-17.  J  une  7,  2005 

■  OSWER  Guidance  9272.0-20.  Dec  21,  2005 

■  Office  of  the  Under  Secretary  of  Defense 
Memorandum  of  April  11,2006 

■  DoD  Instruction  4715.15  Dec  11,  2006 
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UFP-QAPP  Documents 

To  download  documents: 

https://www.epa.aov/fedfac/assurina-aualitv-federal-cleanups 


Manual 


Example  QAPP 


Workbook 


UFP  QAPP 


■  PART  1  is  the  UFP  work  book  manual 

■  Provide  instructions  and  guidance  on  QAPP 
preparation 

■  PART  2A  is  the  blank  forms 

■  The  forms  are  not  mandatory,  but  facilitate  ease 
of  use 

■  PART  2B  is  the  Compendium 

■  Lists  required  QC  activities  for  the  CERCLA  process 
A  minimum  set  of  requirements 

■  Other  Programs  (e.g.  RCRA)  can  use  the 
Compendium  if  agreed  by  all  parties 
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Overview  of  UFP-QAPP  Manual 


Principles 

■  Applicable  to  an v environmental  program 

■  Minimum  QA/QC  activities  -  QA/QC  Compendium 

■  Documentation,  detail  and  effort  varies  by  project 

■  Recommends  use  of  worksheets 

■  Cross  referencing  to  other  documents 

■  Does  not  require  rewrite  of  any  current  QAPP 

Applicable  to: 

■  Investigation 

■  Remediation  activities  or  remedy  solutions 

■  Final  Clean-up  and  long  term 
monitoring/stewardship  activities 
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Format 


■  Follows  Systematic  Planning  Process 
(SPP) 

■  Formal  DQO  Process  (EPA  QA/G-4)  or  other 

■  Fill-in-the-blank  worksheets  for  each  QAPP 
element  from  project  team  decisions 

■  Allows  for  graded  approach 

■  Amount  of  documentation  and  detail  will 
depend  on  complexity  and  scope  of  project 


Quality  Assurance  Project  Plan 
(QAPP) 


■  Integrates  technical  and  quality  control 
aspects  of  a  project  including  ... 

■  planning,  implementation,  assessment,  and 
corrective  actions 

■  Documents  steps  taken  to  ensure 
environmental  data  are  ... 

■  of  the  correct  type  and  quality  required  for  a 
specific  decision  or  use 
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L  QAPP  (Continued) 

■  Organized  and  systematic  description 
of ... 


■  quality  assurance  (QA) 

■  quality  control  (QC) 

■  application  to  the  collection  and  use  of 
environmental  data 
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Other  Documents 


■  Work  Plans,  Health  &  Safety,  Sampling  and 
Analysis  Plans 

■  Can  be  'stand-alone'  documents  or  incorporated 
into  the  UFP-QAPP  using  the  graded  approach 

■  NAVFAC  Sampling  and  Analysis  Plan  is  the  UFP 
QAPP 

■  Work  Plan  refers  to  QAPP 

■  Munitions  and  Chemical  Templates  available 

■  At  a  minimum,  other  documents  must  be 
REFERENCED  in  the  UFP-QAPP 
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UFP-QAPP  Process  Elements 


Scoping  Sessions 
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IDQTF  Current  Activities 


■  Worksheet  streamlining 

■  MMRP  implementation 

■  Advanced  Geophysical  Classification  (AGC) 
Template  finalized 

■  MMRP  UFP  QAPP  Template  under 
development  to  combine/ standardize 
USACE/NAVFAC  templates 

■  Looking  into  UFP  QAPP  templates  for 
emerging  contaminants 
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Review 


What  is  Systematic  Planning? 

■  Group  effort  to  balance  cost  vs.  amount  of  data 
needed  to  make  decision 

■  Understand  how  the  data  will  be  used 

■  Ensure  you  get  what  you  pay  for  by  defining 
project  needs  in  detail 

■  Knowing  what  was  &  was  not  delivered  and  why 


Better  planning  may  add  cost  at  the 
beginning  of  my  project 


Questions  ?? 


Doug  Maddox 

USEPA  Federal  Facilities  Restoration  and  Reuse  Office 
maddox.doua@epa.gov 

202-564-0553 
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Navy  Training 
UFP-QAPP 


John  M.  Jackson 

Geophysicist 
USACE-  SPK/EMCX 


US  Army  Corps  of  Engineers 

BUILDING  STRONG 


Uniform  Federal  Policy 
Quality  Assurance  Project  Plan 

(UFP  QAPP) 


■  UFP-QAPP  is  mandatory  for  all  DERP 
(MMRP  and  IRP)  funded  environmental 
projects 

■  Requires: 

►  The  Right  Technical  Team 

►  Systematic  Planning 

►  Development  of  Clear  Data  Quality  Objectives 

►  Defines  Quality  Checks  before  we  start  Field 
Work 


BUILDING  STRONG, 


UFP  QAPP 


is  a: 

SAMPLING  &  ANALYSIS  PLAN  (SAP) 

SAP  =  FSP  +  QAPP 

IT’S  A  WORK  PLAN!! 


BUILDING  STRONG, 


UFP-QAPP  Documents 


To  download  documents: 

http://www.denix.osd.mil/edaw/Documents.cfm 


Manual 


Workbook 


AGCMR  QAPP 
Template 


Example  QAPP 


BUILDING  STRONG, 


Worksheet  Templates 


■  Prompt  Planning  Process 

■  Organized  by  Process 
Elements 

►  WS’s  1-8:  Management 

Background,  PDT  Lead 
Agency,  Training 
Requirements,  Proof  of  Review, 
and  Approval  Pages 

►  WS9:  Documents  ALL  Project 
Meetings  and  Agreements 

►  WS’s  10-16:  Project  Objectives 

CSM  &  DQO  Development,  and 
Performance  Objectives. 


►  WS’s  17-30:  Design  and  Data 
Collection,  and  Quality  Control 
Requirements 

►  WS’s  31-33:  Assessment  and 

Oversight 

►  WS  34-37:  Data  Review 


BUILDING  STRONG, 


Before  You  SPP  (whole  team) 

■  Take  to  First  Scoping  Meeting: 

•  Management  and  Organization  Worksheets 

>  Worksheets  1  -  8 

•  Planning  Worksheets 

[>  10:  CSM 

t>  1 1 :  Data  Quality  Objectives 

t>  12:  Measurement  Performance  Criteria 

>  13:  Secondary  Data;  Use  and  Limitations 

t>  15:  Project  Action  Limits:  Usually  a  Chemistry  Parameter. 

o  For  MMRP  --  Land  Use  Depth. 

>17:  Study  Design  and  Rational 

>  22:  QC,  Equipment  Testing,  Inspection 

>  14  &  16:  Tasks  and  Schedule 

•  Document  Agreements  /  Discussions  in  /orksheet  9! 


BUILDING  STRONG, 


Organization  Structure:  Safety 


Lines  of  Authority  -  Lines  of  Communication  ——————— 

Figure  3-2:  Explosives  Safety  Operations  Organizational  Structure 


BUILDING  STRONG, 


Organization  Structure:  Geophysics 


Lines  of  Authority  -  Lines  of  Communication  ————————— 

Figure  3-1:  Advanced  Geophysical  Classification  Organizational  Structure 


*11X0  expertise  is  required  to  make  sure  theTOIP  which  can  range  from  intact  munitions  to  sub-components  or  fragments  with  residual 
explosive  and/or  chemical  constituents,  are  defined. 


BUILDING  STRONG, 


Worksheets  1-8:  Lessons  Learned 

■  Section  1 .3.5  of  the  UFP  QAPP  Manual, 

“...all  lead  organization's  project  personnel  are  to  be  listed 
and  roles  must  be  assigned  and  sign  off  on  those  rolls...” 

■  Section  2.3.1 ,  “key  personnel  includes  those  working 
for  the  lead  organization” 

■  Section  2.4.1  supports  inclusion  of  the  lead  agency 
personnel,  including  data  review  personnel. 

►  DoD  (not  contractor)  is  the  lead. 

►  Include  QA  technical  team  members,  specifically  the 
geophysicist,  chemist,  and  OESS  in  org  structure. 

►  Include  Contractor  field  team  leads  and  QC,  not  just 
Contractor  PM,  Geophysicist  and  OESS,  unless  they  are 
the  field  team  leads. 


BUILDING  STRONG, 


Worksheet  #6-  Communication  Pathways 


\\\ 


Communication  Driver 

Initiator 

(name,  title/role, 
and  contact  info) 

Recipient 
(name,  title/role, 
and  contact  info) 

(timingjj^V^ay^  documentation) 

Regulatory  agency  interface 

Name,  DoD  RPM 

Phone/email 

Name,  Organization 
Phone/email 

USACE  Pf\?Tpro^^!^l^K!y^pfbject  update  memorandum  to 

Ffegulato 

Stop  work  due  to  safety 
issues 

Name,  Contractor 

SUXOS 

Phone/email 

Name,  Contractoi^^ 

\l 

Phone/erfrail 

As  possible  following  discovery,  SUXOS  informs  Contractor 

phone  of  critical  safety  issues  and  generates  follow-up  Stop 
“^/pcklvie  mora  nd  u  m 

Minor  QAPP  changes  during 
project  execution 

Major  QAPP  changes  during 
project  execution 

N  a  m  e^Coptfactor  ^ 

W^DSAfE  PM 

■^hon  e/email 

Within  24  hours,  Contractor  PM  submits  field  change  request  form 
to  USACE  PM  for  approval.  Following  approval,  USACE  PM  informs 
regulator  via  email. 

Mobilization  and  surface^  v 
clearance  activitjes^re 
complete^  ' 

^Ptlone/email 

Name,  USACE  QA& 
Safety 

Phone/email 

Upon  completion  of  surface  clearance  activities,  the  SUXOS  informs 
the  USACE  QA  &  Safety  via  Surface  Clearance  Memorandum 

Field^pfogre 

Name,  Contractor 

PM 

Phone/email 

Name,  USACE  PM 
Phone/email 

At  end  of  each  day  of  field  work.  Contractor  PM  provides  daily  QC 
reports  to  USACE  PM  via  email 

in 


BUILDING  STRONG, 


WS  10:  Conceptual  Site  Model  (CSM): 


(UFP-QAPP  Manual  Section  2.5.2) 
(EPA  21 06-G-05  Section  2.2.5) 


Background  -  description,  history, 
including  key  physical  aspects  (e.g., 
site  geology,  hydrology,  topography, 
climate); 


Narrative  history  supported  by 

►  Figures  /  Graphic  Model 
►Tables 


Sources  -  Range  type,  known  or 
suspected  munitions  (MEC/MC); 


■Updated  as  new  data  are  collected. 

■Multiple  sites,  unique  problems, 
-^separate  CSM  for  each 

■ Data  gags  and,  uncertainties  in 

the  CSM  clearly  identified. 


Release  Mechanism  -  including  fate 
and  transport  considerations; 

Land  use  &  Receptors  /  Pathways  - 

potential  for  exposure; 

Model  -  nature  and  extent 
(Consider  Air-to-ground,  vs  artillery 
fan,  vs  multi-  use  ranges,  etc.) 
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BUILDING  STRONG, 


How  to  Use  WS  #  1 0? 


■  Identify  Data  Gaps 

►  NOTE:  data  gap  does  not  =  data  need 

■  Present  What  We  Know 

►  Known  or  Suspected  Munitions  (Specifically) 

►  Confirmed  as  Practice?  Training?  Live? 

■  Define  What  We  Need  to  Know 

►  Can  we  define  an  unacceptable  risk? 

►  What  other  data  will  we  need  to  make  these 
decisions? 


BUILDING  STRONG, 


WS  #  10:  CSM 

Narrative  Supported  by  Graphic  Model 


■  Background  Information, 

►  i.e.  Site  History 

■  Define  the  MRS 

►  Acreage  &  Map 

■  Primary  Release  Mechanisms 
for  MEC/MC 

►  Ground  to  ground 

►  Air  to  ground 

►  Burial... etc. 

■  Findings  from  Previous  Studies 

■  Fate  &  T ransport 


Sources  of  Known  or  Suspected 
MEC/MC 

►  Types  of  Known/Suspected 
MEC/MC 

►  Anomaly  Distribution 

•  Are  there  Concentrated 
Munitions  Use  Areas 
(CMUAs)? 

•  Can  we  define  Depth  of 
Distribution? 


BUILDING  STRONG, 


Worksheet  #  1 0  Cont’d 


■  Key  physical  aspects  ■  Land  use  considerations 


►  Geology  -  background? 

►  Accessibility 

►  Topography /Terrain 

►  Ownerships/  ROEs,  Parcel 

►  Vegetation 

Map 

►  Runoff/Groundwater 

►  Potential  receptors  and 

exposure  pathways 

•  Man  hours 

•  Depth  of  use,  specific  site 
activities  and  users 

►  Surrounding  Use 

►  Existing  Deed  Restrictions  or 
Controls 


BUILDING  STRONG, 


ional  Report  on  Characterization: 
Horizontal  Distribution 


All  Other  Areas 
(MRS  3) 


1  ■■ 

:.'i  ■  >,  ■  ■  \ 

'  . 

\  ■  ■  , 


V  -ri:- Y  *  ■': 


Suspected  Target  Area  #1 
(MRS  1) 


Suspected  Target  Area  #2 
(MRS  2) 


This  example  shows  209  grids  (50’  x  50’)  in  MRS  3. 


BUILDING  STRONG, 


Depth  (inches) 
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?5M^ 

•  Fuzes/Boosters 
o  MD 

•  20MM 

•  Low  Order  HE 

•  Partial  HE  Projectile 
PD  Element 


Worksheet  #  9  Project  Scoping  Session 
Participants  &  Agreements 


Project  Name: 

Site  Name: 

Projected  Date(s)  of  Sampling: 
Project  Manager: 

Site  Location: 

Date  of  Session: 

Scoping  Session  Purpose: 


Name 

Title 

Affiliation 

Phone 

E-mail 

Address 

Project 

Role 

i  Inn  imont 

•  lorn 

imont 

I  inm  imont 


BUILDING  STRONG, 


So  the  PDT  Agrees  to  the  CSM 


■  Document  Decisions  in  Worksheet  #9 


so  Now  Let's 

V>QO\\\ 


BUILDING  STRONG, 


Systematic  Planning:  Developing  DQOs 
>  Uses  what  we  know  about  the  Site 

►  Develop  Plan  for  Additional  Data  Needs 

>What  kind  of  data  do  we  need? 

►  During  Characterization? 

►  During  Remedial  Action? 


BUILDING  STRONG, 


Understanding  DQOs 

■  Who  will  use  the  data? 

■  What  will  the  data  be  used  for? 

■  Phase  of  Data  Acquisition? 

►  Remedial  Investigation? 

•  Characterization 

►  Remedial  Action? 

•  Support  Confidence  &  Achievement  of  RAO 


BUILDING  STRONG, 


SO,  We  Have  the  CSM 


Worksheet  10:  CSM  Worksheet  1 1 :  DQOs 


■  Develop  the  Story 

■  Lead  into  the  Problem 

■  Supports  Need  for 
Further  Remedial 
Response 


Step  1 :  Problem  Statement 

Step  2:  Goals  of  the  Study 

Step  3:  Information  Inputs 

Step  4:  Boundaries 

Step  5:  Analytic  Approach 

Step  6:  Performance 
Objectives 

Step  7:  Detailed  Plan  to 
Obtain  Data 


BUILDING  STRONG, 


How  to  Use  WS  #1 1  ? 


7  Step  DQO  Process 

►  Template  Prompts  each  of  7  steps. 


First,  see  the  template. 


BUILDING  STRONG, 


WS  #  1 1  DQOs:  Steps  1  -4 


(UFP-QAPP  Manual  Section  2.6.1) 

(EPA  21 06-G-05  Section  2.2.6)  3. 

Document  project  quality  objectives  (PQOs)  or 
data  quality  objectives  (DQOs)  using  a  systematic 
planning  process  (SPP) 

EPA's  7  Step  DQOs:  4. 

1.  State  the  Problem.  -  consistent  with  the 
CSM  (data  gaps). 

2.  Identify  the  Goals  of  the  Study,  specific 
study  questions  with  alternative  outcomes. 

(Explain  goals  in  terms  of  how  the  data  will 
be  used  to  choose  among  the  stated 
outcomes.) 


Identify  Information  Inputs. 

a.  Specify  the  types  of  data  required  to  fill 
data  gaps. 

b.  Explain  in  specific  terms  how  all  data 
will  be  used. 

Define  the  Boundaries  of  the  Study. 

a.  Specify  the  target  population  and 
characteristics  of  interest,  define  spatial 
/temporal  limits  and  the  scale  of  inference 
(i.e.,  which  populations  will  be  represented 
by  which  data.) 
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BUILDING  STRONG, 


DQOs  Step  1 


End  of  the  Rl: 

■  Extent 

■  Nature 

■  Unacceptable  risk? 

During  Feasibility  Study: 

■  Develop  the  RAO 

■  Assess  Paths  to  RC 


The  Problem 


End  of  the  RA: 

■  Verify  CSM  assumptions 

■  Confidence  in  Remedial 
Action 

■  Support  Achievement  of 
RAO 


BUILDING  STRONG 


Difference  in  Planning  for  Rl  &  RAs 


Remedial  Investigation  Remedial  Action 


■  Determine  Extent 

►  Vertical  &  Horizontal 

■  Land  Use 

►  Accessibility 

►  Site  Activities 

■  Munitions 

►  Sensitivity 

►  Severity 

■  Determine  Risk 

►  Acceptable  vs 
Unacceptable 


■  Remediate!! 

►  Meet  the  RAO 

■  Confirm  CSM 


BUILDING  STRONG, 


The  UFP  QAPP  Template  for  AGC: 
Remedial  Investigation  vs.  Remedial  Action 


Rl  Template 


RA  Template 


AGC-QAPP 
Revision  Number: 
Revision  Date: 


INTERGOVERNMENTAL  DATA  QUALITY  TASK  FORCE 


Uniform  Federal  Policy 
For 

Quality  Assurance  Project 
Plans 

Advanced  Geophysical  Classification  for 
Munitions  Response 

(AGC-QAPP) 

Version  1.0,  March  2016 


AGCMR  QAPP 
Template 


BUILDING  STRONG, 


So  We  Have  an 
Unacceptable  Risk 


BUILDING  STRONG, 


And  We  Have  the 
Decision  Document 


BUILDING  STRONG, 


DQO  Steps  1  &  2: 

Remedial  Action 

■  Support  Achievement  of  RAO 

■  Support  Confidence  in  Remedial  Action 

►  Depth  of  Detection/Retrieved  vs  Land  Use  Depth 

►  Verify  Horizontal  and  Vertical  Distribution 

■  Achieve  an  Acceptable  End  State 

►  Determine  if  remedy  (UU/UE  or  other)  can  be 
supported  (MQO/DQOs  met  and  CSM  confirmed) 

►  Determine  if  additional  response  required  (CSM 
or  DQOs  were  not  achieved;  maybe  ESD?) 


BUILDING  STRONG, 


From  The  AGCMR  QAPP 


Step  1:  State  the  Problem.  Define  the  problem  that  necessitates  the  study.  Examine  budget  and 
schedule  issues. 

Site-specific  problem  statement;  {Example)  Previous  investigations  (list)  have  indicated  that  MEC  in  the 

form  of  DMM  and  UXO  including  (x,  y,  and  z)  are  present  at  site _ ,  resulting  from  its  use 

between  (years) _ and _ as  a  [describe  the  type  of  facility  and  its  uses).  As  shown  in  the 

CSM  these  materials  present  a n|unacceptablejiskjf r o m  explosive  hazards  to  {describe  current  receptors 
and  potential  future  receptors  based  on  anticipated  land  use.) 

Advanced  geophysical  classification  uses  advanced  sensors  and  geophysical  classifiers  to  estimate 
physical  properties  of  the  item  {e.g.,  depth,  size,  aspect  ratio,  wall  thickness,  symmetr/)  and  determine 
whether  the  item  is  a  TOI  (i.e.,  highly  likely  to  be  MEC)  or  non-TOI  (i.e.,  highly  unlikely  to  be  MEC).  Using 
this  information  in  a  structured  decision-making  process,  project  teams  will  be  able  to  make  informed 
decisions  about  whether  an  item  should  be  excavated  or  can  be  left  in  place. 


BUILDING  STRONG, 


Worksheet  1 1 :  Step  2,  Goals  of  the  Study 


Step  2i  Identify  the  goals  of  the  data  collection.  State  how  data  will  be  used  in  meeting  objectives  and 
solving  the  problem.  Identify  study  questions,  including  RA  objectives.  Define  alternative  outcomes. 

Identify  the  principal  study  question:  (Example)  Based  on  current  and  anticipated  future  land  use 
scenarios,  which  detected  buried  metal  objects  must  be  removed,  and  which  ones  may  be  left  it  place? 

Identify  alternative  outcomes:  (Example)  To  classify  an  object  as  a  TOI  and  remove  it,  or  to  classify  it  as 
non-TOI  and  leave  it  in  place. 

State  how  the  data  will  be  used  in  solving  the  problem:  (Example)  Advanced  geophysical  classification 
will  be  used  to  1)  detect  anomalies  resulting  from  DMM,  UXO,  and  other  metallic  debris  and  2)  classify 
anomalies  so  that  informed  decisions  can  be  made  as  to  whether  the  anomaly  results  from  a  TOI  that 
should  be  removed,  or  a  non-TOI  that  may  be  left  in  place.  Geophysical  data  collected  using  advanced 
EMI  sensors  in  a  dynamic  mode  will  be  used  to  initially  detect  and  document  the  locations  of  subsurface 
anomalies.  Geophysical  data  collected  using  advanced  EMI  sensors  in  a  cued  (static)  mode  will  then  be 
used  to  classify  each  anomaly  as  follows:  l)TOI,  i.e.,  highly  likely  to  be  DMM  or  UXO;  2)  Non-TOI,  i.e., 
highly  unlikely  to  be  DMM  or  UXO;  or  3)  Inconclusive,  Detected  items  classified  as  "TOI"  and 
"inconclusive"  will  be  targeted  for  removal,  items  classified  as  non-TOI  will  be  left  in  place.  The  results 
of  geophysical  detection  and  classification  and  the  subsequent  intrusive  investigation  must  meet 
established  DQOs  to  allow  the  anticipated  land  reuse  to  take  place  after  the  removal  of  TO!. 


BUILDING  STRONG, 


DQOs  Step  3:  Information  Inputs 


■  What  type  of  data  are  needed? 

►  Magnetometer,  EMI,  digital  geophysical  mapping 

►  Detection  capabilities  meet  land  use  depth?? 

■  Data  Gaps 

►  True  Vertical  and  Lateral  Extent 

►  Detection  Depth  Achieved 

►  Background  S/N  ratio 


BUILDING  STRONG, 


Data  Needs  for  the  Remedial  Action 


■  Boundaries  (resulting  from  Rl,  Defined  in  DD) 

■  Remedial  Action  Objective 

►  Contaminant  &  Media  (from  Rl) 

►  Depth,  to  Protect  Specific  Users  (from  Rl) 

►  The  Objective 

■  Specific  Type  of  Munitions 

►  Size,  -Detection  Limits 

►  Sensitivities,  Severities 


BUILDING  STRONG, 


DQOs  Step  3:  Information  Inputs 


■  The  AGC  QAPP 


Step  3:  Identify  information  inputs.  Identify  data  and  information  needed  to  answer  the  study 
questions. 

(Example) 


is  comprehensive 

►  Detection 
Components 

►  Cueing 
Components 

►  Intrusive 
Components 


•  Up-to-date  CSM  summarizing  site  conditions  based  on  previous  studies  (e.g.,  Preliminary 
Assessment  (PA),  Site  Inspection  (SI)  and  Remedial  Investigation/Feasibility  Study  (RI/FS)).  [See 
Worksheet  #10]: 

•  Detection  survey  results,  including: 

o  Areas  covered 
o  System  QC  test  results 
o  Instrument  Verification  Strip  (IVS)  results 
o  Surveyed  validation  seed  and  QC  seed  locations 
o  Anomaly  detections  and  responses 
o  Data  analysis  results,  including 

■  Anomaly  locations 

■  Unique  anomaly  identification  numbers 

■  Z-component  amplitude  and  dipole  response  for  each  anomaly 

■  Detection  survey  data  validation  report 

■  Detection  survey  data  usability  evaluation 

■  Updated  CSM 

•  Cued  survey  results,  including: 

o  System  QC  results 
o  IVS  results 
o  Background  data 

o  Surveyed  validation  seed  and  QC  seed  locations  and  types 
o  Unique  anomaly  identification  numbers  and  locations 
o  Site-specific  munitions  library 

o  Definition  of  items  representing  unacceptable  explosive  hazard 
o  Classification  of  anomalies  with  confidence  metric 
o  Cued  survey  data  validation  report 
o  Cued  survey  data  usability  evaluation 


Etc. 


BUILDING  STRONG, 


Worksheet  #11 :  Step  4  -  Boundaries 


BUILDING  STRONG, 


From  The  AGCMR  QAPP 


Step  4:  Define  the  boundaries  of  the  project.  Specify  the  target  population  and  characteristics  of 
interest.  Define  spatial  and  temporal  boundaries.  {Discuss  NAOC  Comment  129] 

Target  population:  {Example}  The  target  population  for  this  study  includes  the  following  M EC  confirmed 
or  suspected  to  exist  in  the  study  area: 


Table  11-1:  Target  Population 


Confirmed  Munitions 
(including  nomenclature,  if 
known) 

MEC  Type 
(UXO,  DMM,  or 
both} 

Munition 

Length 

Observed  Depth 
of  Penetration 
(to  center  of 
mass} 

Expected  i 

Detection 

Threshold 

37mm  (unknown  mark/mod) 

UXO 

75mm  (unknown  mark/mod) 

UXO 

Suspected  Munitions 
(including  nomenclature,  if 
known) 

MEC  Type 
(UXO,  DMM,  or 
both} 

60  mm  mortar,  M49A3 

UXO 

155mm,  M107 

UXO 

Characteristics  of  interest:  {Example}  The  characteristics  of  interest  are  those  characteristics  (e.g.,  size, 


symmetry,  aspect  ratio,  object  density,  and  wall  thickness)  that  will  allow  classifiers  to  determine 
whether  an  anomaly  is  a  TO!  or  non-TOI. 


STRONG^ 


DQO  Steps  1  -4 


Lots  to  Digest  here... 

...and  not  even  finished  with  WS  #1 1 ! 
(We  still  have  DQO  steps  5-7...) 


BUILDING  STRONG, 


Data  Quality  Objectives:  Steps  5-7 


UFP  QAPP  Worksheet  #11 : 
(UFP-QAPP  Manual  Section  2.6.1) 
(EPA  2106-G-05  Section  2.2.6) 

5  .  Develop  the  Project  Data  Collection 

and  Analysis  Approach. 

Define  the  parameter(s)  of  interest,  and 
develop  the  logic  or  statistics  for 
drawing  conclusions  from  findings. 

(What  kind  and  how  much  data?) 


6.  Specify  Performance/Acceptance 

Criteria. 

Develop  performance  criteria  (for  new 
data  being  collected)  or  acceptance 
criteria  (for  existing  data  being  considered 
for  use). 

Focus  on  WS  #  12  &  22 


For  decision  problems  -“if— then”  7  Develop  Detailed  Plan  for  Obtaining  Data 

statements,  or  decision  rules,  to  link  VVS  #-\-\  overview  of  sampling  design 
potential  results  with  outcomes.  WS  #1 7  -design  details. 

WS  #19,  20,  24-28,  and  30- 

analysis  design  requirements. 
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BUILDING  STRONG, 


From  The  AGC  QAPP 


Cued  Phase 

Parameters  of  interest:  (Example)  Spatial  extent  of  detected  anomaly,  cued  measurement  SNR, 
inversion  fit  coherence,  and  inversion  outputs  of  (31,  (3  2,  (33,  x,  y,  and  z. 

Type  of  inference:  (Example)  If  any  of  the  following  three  criteria  are  met,  the  anomaly  will  be  selected 
as  a  TO!:  1)  the  polarizability  matches  (within  specifications  established  on  Worksheet  #22)  that  of  an 
item  in  the  project-specific  TOI  library,  2)  estimates  of  the  size,  shape,  symmetry,  and  wall  thickness 
calculated  from  the  polarizability,  indicates  the  item  is  long,  cylindrical,  and  thick-walled,  or  3)  there  is  a  3 
group  (cluster)  of  x  or  more  anomalies  having  similar  polarizabilities  that,  after  investigation,  are 
discovered  to  be  TOI.  Anomalies  with  poor  inversion  fit  coherence  that,  after  considering  all  available 
information,  cannot  be  ruled  as  non-TOI  (i.e.,  the  data  are  inconclusive)  will  be  added  to  the  TOI  list. 


Decision  rules:  (Examples) 


If  all  or  a  portion  of  the  study  area  is  determined  to  have  an  anomaly  density  too  high  for  cued 
analysis,  then  an  alternative  approach  will  be  developed  (factors  for  evaluating  anomaly  density 
are  discussed  in  Worksheet  #17). 

If  the  object  is  classified  as  TOI  (highly  likely  to  be  a  munition),  then  the  object  will  be  excavated. 
If  the  object  is  classified  as  non-TOI  (highly  unlikely  to  be  a  munition),  then  the  object  will  be  left 
in  place. 

If  the  object  is  classified  as  inconclusive,  then  the  object  will  be  excavated. 


BUILDING  STRONG, 


Data  Quality  Objectives:  Steps  5-7 


UFP  QAPP  Worksheet  #11 : 
(UFP-QAPP  Manual  Section  2.6.1) 
(EPA  2106-G-05  Section  2.2.6) 

5  .  Develop  the  Analytic  Approach. 

Define  the  parameter(s)  of  interest,  and 
develop  the  logic  or  statistics  for 
drawing  conclusions  from  findings. 


6.  Specify  Performance/Acceptance 

Criteria. 

Develop  performance  criteria  (for  new 
data  being  collected)  or  acceptance 
criteria  (for  existing  data  being  considered 
for  use). 

Focus  on  WS  #  12  &  22 


(What  kind  and  how  much  data?) 


For  decision  problems  -“if— then” 

statements,  or  decision  rules,  to  link  Develop  Detailed  Plan  for  Obtaining  Data 

potential  results  with  outcomes.  WS  ^  overview  of  sampling  design 

WS  #1 7  -design  details. 

WS  #19,  20,  24-28,  and  30- 

analysis  design  requirements. 
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BUILDING  STRONG, 


WS  11: 


Step  5:  Develop  Approach 

■  Advanced  Classification 

►  Discuss  Applicability 

►  Benefits 

►  Limitations 


Steps  6 


Step  6:  Measurement 
Performance  Criteria 


BUILDING  STRONG 


Worksheet  #12 


Measurement  Performance  Criteria 

■  Quantitative  measurement  of  data  quality. 

■  Tabulated  for  the  following  activities. 

►  General  measurement  performance  criteria 

►  Geophysical  Instrument  functionality 

►  Positioning  instrument  functionality 

►  Data  collection 

►  Data  processing 

►  Auditing  procedures 

►  Other  project  requirements 


Data  Quality  Indicators 


BUILDING  STRONG, 


Data  Quality  Indicators 


Coo d  occur, icy  Poor  accuracy  Poor  accuracy 

Cloud  precision  CikxI  precision  Poor  precision 


■  Precision 

■  Accuracy 

■  Representativeness 

■  Completeness 


Comparability 

Sensitivity 


BUILDING  STRONG, 


Example,  Develop  Performance  Criteria  for 
Advanced  Geophysical  Classification 


■  DQI’s  addressed  for  each  measurement 
phase/activity 

■  Three  phases  in  AGC: 

►  Detection  (Dynamic)  Survey 

►  Classification  (Cued)  Survey 

►  Intrusive  Survey 


Phases  for  Advanced  Geophysical  Classification 


3  Phases  of  AGC  Data 

a.  Dynamic 

b.  Cued 

c.  Intrusive 


DQI  for  each  Phase 

Precision 
Accuracy 

Representativeness 
Completeness 
Comparability 
Sensitivity 


Worksheet  Development: 


BUILDING  STRONG, 


Dynamic  Survey  QC 


■  Performance  Objectives: 

►  verify  geophysical  sensor  is  operating  properly,  and 

►  provide  ongoing  monitoring  of  the  data  quality. 

■  Two  key  elements: 

►  Instrument  Verification  Strip  (IVS) 

►  Production  Area  Blind  Seeding 

•  Contractor  QC  seeds  and 

•  Blind  QA/validation  seeds 


BUILDING  STRONG, 


Step  6:  Performance  Criteria 


ivs 

■  Confirms  that  the  geophysical 
detection  system  is  operating 
properly 

■  Usually  single  pass  over  an 
IVS  line  before  start  and  after 
completion  of  production  work 
each  day. 


QC  &  Validation  Seeding 

■  Ongoing  monitoring  of  the 
quality  of  data  collection  and 
analysis  as  it  is  performed 

■  Expect  1  -3  per  day,  one  at 
limit  of  detection  depth 


■  QC  seed  failure  results  in  CA 
in  the  Field 

■  QA  seed  failure  results  in  loss 
of  confidence  in  data  and  the 
whole  system 


BUILDING  STRONG, 


IVS 

Endpoint 


IVS  Requirement 


Item  Item  Blank  item  IVS 

m  #2  Space  #3  Endpoint 

< - * - ■ 

~5m  Typical  l\JS  plan 


Site  Noise  Line 


IVS 

Define  Project’s  minimum 
frequency  for  IVS  and 
necessary  criteria  to 
demonstrate  instrument  is 
fully  functional. 


Criteria 

Different  for  Dynamic  vs 
Cued  Surveys 

►  See  Decision  Tree:  WS  17 


BUILDING  STRONG, 


Performance  Testing? 


Surface 


QA  Seeds:  System  Test 


QC  Seeds:  Verify  DL 
Throughout  Fieldwork 

Increased  Reduction  in 

Possibility  of  False  Confidence 

Negatives 


0  a 


High 

Confidence 


<-  ?  Detection 
Limit  (DL) 


What’s  the  Detection  Limit? 
What  Does  It  Mean? 


BUILDING  STRONG, 


Key  Worksheets  for  “The  Plan” 


Measurement 
Performance 
Criteria:  WS  #12 


■  Criteria  that 
collected  data 
must  meet  to 
satisfy  the 
DQOs. 

■  Failure  impacts 
end  uses  of  the 
data. 


Design  /  Workflow: 

WS  #17  and 

Schedule  for  DFW: 

WS  #1 4-1 6 

■  Criteria  that 
collected  data 
must  meet  to 
satisfy  the  DQOs. 

■  Failure  impacts 
end  uses  of  the 
data. 


Testing, 
Inspection  and 
QC:  WS  #22 

■  Defines  Specific 
Acceptance 
Criteria  for  each 
test. 

■  Failure  response 
and  root  cause 
analysis  for  each 
MPC,  each 
phase. 


*DFW=  definable  features  of  work 


BUILDING  STRONG, 


WS  #12:  Measurement  Performance  Criteria  -  Detection 


Measurement 
Performance  Activity  (or 
DFW) 

Data  Quality 
Indicator 

Specification 

Activity  Used  to  Assess 
Performance 

QC  Seeding 

Representativeness 

Blind  QC  seeds  will  be  placed  at  the  site  by  the 
contractor.  Blind  QC  seeds  must  be  detectable  as 
defined  by  the  DQOs  and  located  throughout  the 
horizontal  and  vertical  survey  boundaries  defined  in 
the  DQOs.  [The  blind  seed  plan  should  describe  the 
number  and  types  of  blind  QC  seeds.]  Blind  QC 
seeds  will  be  distributed  such  that  the  field  team 
can  be  expected  to  encounter  between  one  and 
three  seeds  per  day  per  team. 

Review  of  Production  Area  QC  Seeding 
Report 

Detection  Survey 

Completeness 

100%  of  the  site  is  sampled. 

Verification  of  conformance  to 
measurement  quality  objectives 
(MQOs)  for  in-line  spacing  and  cross- 
line  spacing  (see  Worksheet  #22) 

Detection  survey 

Sensitivity 

This  worksheet  must  describe  the  project-specific 
detection  threshold.  (Example)  A  detection 
threshold  of  .1.7  mV/A  and  SNR  .  5  is  required  to 
detect  a  [37  mm  projectile]  lying  horizontally  at  a 
depth  of  [0.3  m]. 

Initial  and  ongoing  Instrument 
Verification  strip  (IVS)  surveys 

Blind  QC  and  validation  seed  detection 
Analysis  of  background  variability 
across  the  site 

Detection  survey 

Accuracy/ 

Completeness 

100%  of  validation  seeds  must  be  detected. 

Review  of  validation  seed  detection 
results  per  survey  unit 

Detection  survey 

Completeness/ 

Comparability 

Complete  project-specific  databases  and  target  lists 
delivered. 

Data  verification/data  validation 

BUILDING  STRONG, 


CFW3 


Figure  17-1:  Advanced  Geophysical  Classification  Decision  Tree 

Preliminary  Tasks  and  Anomaly  Detection  Survey 


WS  #1 7: 
Design 

Work  Flow  Decision  Tree 


Preliminary  Tasks 
& 

Detection  Survey 


BUILDING  STRONG, 


WS  #22:  Testing,  Inspection  and  QC  -  Detection 


Measurement  Quality  Objective 

DFW/SOP 

Reference 

Frequency 

Responsible  Person/ Report 
Method/Verified  by 

Acceptance  Criteria 

Failure  Response 

Verify  correct  assembly 

Once  following  assembly 

Field  Team  Leader/instrument 
assembly  checklist/Project 

As  specified  in  Assembly  checklist 

RCA/CA:  Make 

necessary 

Initial  Instrument  Function  Test 
(TEMTADS) 

(Instrument  response  amplitudes) 

Once  following  assembly 

Field  Geophysicist/ Initial  IVS 
Memorandum/  Project 
Geophysicist 

Response  (mean  static  spike  minus 
mean  static  background)  within  20% 
of  predicted  response  for  all 
transmit/receive  (Tx/Rx)  combinations 

RCA/CA:  Make 
necessary 

adjustments,  and  re¬ 
verify 

Initial  Instrument  Function  Test 
(MetalMapper)  (five  measurements  over  a 
small  ISO80  target,  one  in  each  quadrant  of 
the  sensor  and  one  directly  under  the 
center  of  the  array).  Derived  polarizabilities 
for  each  measurement  are  compared  to  the 
library. 

Once  following  assembly 

Field  Team  Leader/ 1  nstrument 
Assembly  Checklist/  Project 
Geophysicist 

Library  match  metric  >  0.95  for  each 
of  the  five  sets  of  inverted 
polarizabilities 

RCA/CA:  Make 
necessary 

adjustments,  and  re¬ 
verify 

Initial  Instrument  Function  Test 
(EM61) 

Once  following  assembly 

Field  Geophysicist/  Initial  IVS 
Memorandum/  Project 
Geophysicist 

Response  (mean  static  spike  minus 
mean  static  background)  within  20% 
of  predicted  response  for  all  channels 

RCA/CA:  Make 
necessary 

adjustments,  and  re¬ 
verify 

Initial  detection  survey  positioning 
accuracy  (IVS)  [NAOC101) 

Once  prior  to  start  of  detection  survey 
data  acquisition 

Project  Geophysicist/ IVS 
Memorandum/QC  Geophysicist 

Derived  positions  of  IVS  target(s)  are 
within  25cm  of  the  ground  truth 
locations 

RCA/CA:  Make 
necessary 

adjustments,  and  re¬ 
verify 

BUILDING  STRONG, 


WS  #22  (continued-  detection) 


Measurement  Quality 
Objective 

DFW/SOP 

Reference 

Frequency 

Responsible  Person/ 
Report  Method/ 
Verified  by 

Acceptance  Criteria 

Failure  Response 

Ongoing  Instrument 

Function  Test 
(EM  61) 

Beginning  and  end  of 
each  day  and  each 
time  instrument  Is 

turned  on 

Field  Team  Leader/ 
running  QC 
s  u  m  m  a  ry/Project  o  r 

QC  Geophysicist 

Response  (mean  static 
spike  minus  mean 
static  background^ 
within  20%  off 
predicted  response 

for  altl  channels 

CA:  Make  necessary  repairs  and 
re- verify 

Ongoing  dynamic 
positioning  precision 
(IV5) 

Beginning  and  end  of 
each  day 

Project  Geophysicist/ 
running  QC 
summary/QC 
Geophysicist 

Derived  positions  of 

IVS  target(s)  within  25 
cm  of  the  average 

locations 

RCA/CA 

In-line  measurement 

spacing 

(TFMTADS) 

Verified  for  each 

survey  unit  using 
[describe  tool  to  be 
used]  based  upon 
monostatic  Z  coil  data 

positions- 

Project  Geophysicist/ 
runni  n  g  QC  su  m  m  ary/ 

QC  Geophysicist 

100%  <  0.20m 

between  successive 

measurements 

RCA/CA 

CA  assumption:  data  set  failsr 
(recollect  portions  that  fail] 

In-line  measurement 
spacing 

(Metal  Mapper) 

Verified  for  each 
survey  unit  using 
[describe  tool  to  be 
used'  based  upon 
monostatic  Z  coil  data 
positions 

Project  Geophysicist/ 
running  QC  summary/ 

QC  Geophysicist 

100%  i  0.25m 

between  successive 

measurements 

RCA/CA 

In-line  measurement 

spacing 

(EM  Si) 

Verified  for  each 
survey  unit  using 
[describe  tool  to  be 
used]  based  upon 
monostatic  Z  coil  data 
positions 

Project  Geophysicist/ 
runni  n  g  QC  su  m  m  ary/ 

QC  Geophysicist 

100%  <  0.25m 

between  successive 

measurements 

RCA/CA 

M 


BUILDING  STRONG, 


What  SOPs? 


■  Instrument  Assembly 

■  Setup/Continued  Testing 

►  Processing  procedures 

►  How  is  it  tested/calibrated? 

•  Background  Tests 

•  Instrument  Verification  Strip  (IVS) 

>  Initial  Calibration 

•  Continued  Testing  (QC) 

>  QC  Seeds 

•  Quality  Systems  Test  (QA) 

>  QA  Seeds 

■  Data  Use _ 


BUILDING  STRONG^ 


WS  #1 2:  Measurement  Performance  Criteria  -  Cued  Survey 


Measurement 
Performance  Activity  (or 
DFW) 

Data  Quality 
Indicator 

Specification 

Activity  Used  to  Assess 
Performance 

Classification  survey 

Completeness/ 

Comparability 

Library  must  include  signatures  for  all  munitions 
known  or  suspected  to  be  present  at  the  site,  as 
listed  in  the  CSM. 

Verification  of  site-specific  library 

Classification  survey 

Representativeness/ 

Accuracy 

Background  data  will  be  collected  at  least  once 
every  two  hours  of  cued  survey  data  collection. 
Background  locations  will  be  selected  such  that 
background  data  will  be  representative  of  the 
various  subsurface  conditions  expected  to  be 
encountered  within  each  survey  unit  at  the  site. 

Data  verification/data  validation 

Classification  survey 

Completeness 

All  detected  anomalies  classified  as: 

1. TOI 

2.  Non  TOI 

3.  Inconclusive 

Data  verification 

Classification  survey 

Accuracy/ 

Completeness 

Cued  survey  must  correctly  classify  100%  of  all 
validation  seeds. 

Review  of  validation  seed  classification 
results 

Classification  survey 

Accuracy 

100%  of  predicted  non-TOI  that  are  intrusively 
investigated  are  confirmed  to  be  non-TOI. 

Visual  inspection  of  recovered  items 
from  classification  validation 

BUILDING  STRONG, 


cnw  ia  cfw3  dfw^wjs  nrwfl 


Cued  Survey 


WS  #1 7: 
Design 

Work  Flow  Decision  Tree 

Cued  Survey 


BUILDING  STRONG, 


WS  #22:  Testing,  Inspection  and  QC  -  Cued 


Table  22-2s  Cued  Survey  [instruments _ ?  classification  took _ } 


Measurement  Quality 
Objective 

DFW/SOP 

Reference 

Frequency 

r  v.u_-. 

Responsible  Person/ 
Report  Method/ 
Verified  by: 

Acceptance  Criteria 

Failure  Response 

Verify  correct  assembly 

Once  following 
assembly 

Field  Team  Leader/ 
instrument  assembly 
chec  k  list /Project 
Geophysicist 

As  specified  in 
instrument  assembly 
checklist 

R€A/CA\  Ma  ke  necessary 
adjustments,  and  re-uerify 

Initial  sensor  function 
test  (TEMTTADSI 

Once  following 
assembly 

Field  Team  Leader/ 
instrument  assembly 
diecklist/Projedt 
Geophysicist 

Response  jmean  static 
spike  minus  mean 
static  background/ 
within  20%  of 
predicted  response  for 
all  Tx/Hx  combinations 

RCA/CA:  make  necessary 
repairs/ adjustments  and  re¬ 
verify 

Initial  instrument 
function  test 

JMeta Mapper)  Jfive 
measurements  over  a 
sma  1  I50BD  target,  one 
in  each  quadrant  of  the 
sensor  and  one  directly 
under  the  center  of  the 
array).  Derived 
polarizabilrtiesfor  each 
measurement  are 
compared  to  the  library 

Once  following 
assembly 

Field  Team  Leader/ 
instrument  assembly 
checklist/  Project 
Geophysicist 

Library  match  metric  z: 
Q„^5  for  each  of  the 
five  sets  of  inverted 
po  amiabilities 

RCA/CA:  make  necessary 
re  pairs/ adjustments  and  re¬ 
verify 

BUILDING  STRONG, 


WS  #22:  Testing, 


Inspection  and  QC  -  Cued 


Table  22-2:  Cued  Survey  (instrument: _ ;  classification  tool: _ ) 


Measurement  Quality 
Objective 

DFW/SOP 

Reference 

Frequency 

Responsible  Person/ 
Report  Method/ 
Verified  by: 

Acceptance  Criteria 

Failure  Response 

Initial  IVS  background 
measurement  and 
background  verification 
(five  background 
m  ea  s  u  rem  e  nts,  o  n  e 
centered  at  the  flag  and 
one  offset  at  least  Yz 
sensor  spacing  in  each 
cardinal  direction) 

Once  during  initial 
system  IVS  test 

Field  Team  Leader/ 

Initial  IVS 

memorandum/  Project 
Geophysicist 

All  five  measurements 
(decay  amplitude) 
within  the  noise  level 

of  each  other  and 
library  match  from  all 
four  offset 

measurements  >0.9 

RCA/C  A:  reject/replace  BG 
location 

Initial  derived 
polarizabilities  accuracy 
(IVS) 

Once  during  initial 
system  IVS  test 

Project  Geophysicist/ 
Initial  IVS 

memorandum/  QC 
Geophysicist 

Library  Match  metric  > 
0.9  for  each  set  of 

Inverted 
pola  rizabilities 

RCA/CA 

Derived  target  position 
accuracy  (IVS) 

Once  during  initial 
system  IVS  test 

Project  Geophysicist/ 
Initial  IVS 

Memorandum/QC 

Geophysicist 

Ail  IVS  item  fit 

locations  within  0.25m 
of  ground  truth 
locations 

RCA/CA 

Ongoing  derived 
polarizabilities  precision 
(IVS) 

Beginning  and  end  of 
each  day  as  part  of  IVS 
testing 

Project  Geophysicist/ 
tracking  summary/QC 
Geophysicist 

Library  Match  to  initial 
polarizabilities  metric 
>  0.9  for  each  set  of 
three  inverted 
polarizabilities 

RCA/CA 

Ongoing  derived  target 
position  precision  (IVS) 

Beginning  and  end  of 
each  day  as  part  of  IVS 
testing 

Project  Geophysicist/ 
tracking  summary/QC 
Geophysicist 

All  IVS  items  fit 

locations  within  0.25m 
of  average  of  derived 
fit  locations 

RCA/CA 

BUILDING  STRONG, 


WS  #22  (continued-  cued) 


Measurement  Quality 
Objective 

DFW/SOP 

Reference 

Frequency 

Responsible  Person/ 
Report  Method/ 
Verified  by: 

Acceptance  Criteria 

Failure  Response 

Ongoing  instrument 

function  test 
(Metal  Mapper) 

Each  time  instrument 

is  turned  on 

Field  Team  Leader/ 
t  racki  ng  su  mma  ry/ 
Project  Geophysicist 

Response  within  20% 
of  predicted  response 

CA:  Make  necessary  repairs  and 
re-verify 

Transmit  current  levels 
(TEMTADS) 

Evaluated  for  each 

sensor  measurement 

Field  Team  Leader/ 
t  ra  cki  ng  su  mma  ry/ 
Project  Geophysicist 

Current  must  be  >5„5A 

CA:  stop  data  acquisition 

activities  until  condition 

co  rrected 

Transmit  current  levels 
(Metal  Mapper) 

Evaluated  for  each 

sensor  measurement 

Field  Team  Leader/ 
t  ra  cki  ng  su  mma  ry/ 
Project  Geophysicist 

Current  must  be  £G_5  A 

CA:  stop  data  acquisition 

activities  until  condition 

co  rrected 

Confirm  all  background 

measurements  are  valid 

Evaluated  for  each 

background 

measurement 

Project  Geophysicist/ 
Background  summary/ 
QC  Geophysicist 

Ensure  background 

variation  does  not 

impact  ability  to 
classify  correctly 

CA:  BG  measurement  rejected 

and  removed  from  active  BG 

measurements 

Confirm  adequate 
spacing  between  units 
(TEMTADS) 

Evaluated  at  start  of 
each  day  (or  grid) 

Field  Team  Leader/ 

Field  Logbook/ 

Project  Geophysicist 

Minimum  separation 

of  50  m 

CA::  Recollect  all  coincident 

measurements 

Confirm  adequate 
spacing  between  units 
(Metal  Mapper) 

Evaluated  at  start  of 
each  day  (or  grid) 

Field  Team  Leader/ 

Field  Logbook/ 

Project  Geophysicist 

Minimum  separation 
of  25  m 

CA:  Recollect  all  coincident 

measurements 

Confirm  inversion  model 
supports  classification  (1 
of  3.) 

Evaluated  for  all 

models  derived  from  a 
measurement  (i.e, 
single  item  and  multi¬ 
item  models) 

Project  Geophysicist/ 
Measurement 

QC  summary/ 

QC  Geophysicist 

Derived  model 
response  must  fit  the 

observed  data  with  a 

fit  coherence  >  O.R* 

Follow  procedure  in  SOPor 
RCA/CA 

Confirm  inversion  model 
supports  classification  (2 
of  3) 

Evaluated  for  derived 

target 

Project  Geophysicist/ 
Measurement 

QC  summary/ 

QC  Geophysicist 

Fit  location  estimate 

of  item  i  0.4m  from 

center  of  sensor 

Follow  procedure  in  SOP  or 
RCA/CA 

m 


BUILDING  STRONG, 


WS  #12:  Performance  Criteria  -  Intrusive 


Measurement 
Performance  Activity  (or 
DFW) 

Data  Quality 
Indicator 

Specification 

Activity  Used  to  Assess 
Performance 

Intrusive  Investigation 
(classification  validation) 

Accuracy 

Inversion  results  correctly  predict  one  or  more 
physical  properties  (e.g.  size,  symmetry,  or  wall 
thickness)  of  the  recovered  items  (specific  tests 
and  test  objectives  established  during  project 
planning). 

Visual  inspection  and  qualitative 
evaluation  of  items  recovered  during 
classification  validation 

Intrusive  Investigation 

Completeness/ 

Comparability 

Complete  project-specific  database  including 
records  reconciling  inversion  results  to  the  physical 
properties  of  the  recovered  items. 

Data  Verification 

Data  Validation 

BUILDING  STRONG, 


PFW13  EFW 12  CFWfli 


Intrusive  Investigation 


WS  #17: 
Design 

Work  Flow  Decision  Tree 
Intrusive  Investigation 


BUILDING  STRONG, 


WS  #22:  Testing,  Inspection  and  QC  -  Intrusive 


Measurement  Quality  Objective 

DFW/SOP 

Reference 

Frequency 

Responsible  Person/  Report 
Method/Verified  by 

Acceptance  Criteria 

Failure  Response 

Confirm  derived  features  match  ground 
truth  (1  of  2) 

Evaluated  for  all  recovered  items 

Project 

Geophysicist/Measurement  QC 
Summary  or  intrusive 

100%  of  recovered  (excluding 
inconclusive  category)  item  positions 
<  0.25m  from  predicted  position  (x, 

RCA/CA 

Confirm  derived  features  match  ground 
truth  (2  of  2) 

Evaluated  for  all  recovered  items 

UXO  Dig  Team/ Dig  List  and 
intrusive  database/Project  or  QC 
Geophysicist 

100%  of  recovered  object  size 
estimates  (excluding  inconclusive 
category)  qualitatively  match 

RCA/CA 

Verification  of  TOI/non-TOI  threshold 

Dig  200  anomalies  beyond  last  TOI  on  Dig 
List 

Project  Geophysicist/  Verification 
and  Validation  Report/QC 

100%  of  predicted  non-TOI  intrusively 
investigated  are  non-TOI 

RCA/CA.  Adjust 
threshold 

Classification  validation 

Random  selection  of  200  non-TOI 

and  Validation  Report/  QC 
Geophysicist 

qualitatively  matches  predicted 
size/shape 

RCA/CA.  Document  in 

DUA 

BUILDING  STRONG, 


Worksheets  14  &16 


■  Tabulate  Project  Tasks  from  Flow  Diagram 

►  Activity 

►  Responsible  Party 

►  Planned  Start  Date 

►  Planned  Completion  Date 

►  Deliverable 

►  Deliverable  Due  Date 


BUILDING  STRONG, 


Oversight  and  Assessment:  WS  #29 


■  Data  Management  Tasks 

►  Documents  to  be  Generated 

►  Frequencies 

►  All  Record  Location  &  Durations 

►  Distributions 


Field  Team  Generated  &  Reported 


BUILDING  STRONG, 


Oversight  and  Assessment:  WS  #31 ,  (32  &  33) 
■  Assessment  &  Corrective  Actions:  WS  #31 


►  Assessment  Type  for 

•  Preparation 

•  Initial  Setup 

•  Ongoing 

►  Responsible  Party 

•  For  Review 


►  Frequency 

►  Assessment  Deliverable 


•  Checklist  /  Summary  of  Findings 


►  Due  Date 


QC  Reviewed  &  Reported 


BUILDING  STRONG, 


Oversight  and  Assessment:  WS  #31 ,  (32  &  33) 
■  Assessment  &  Corrective  Actions:  WS  #31 

►  Assessment  Type  for 

•  All  Phases 

►  Responsible  Party  for  Response 

►  Timeframe  for  Response 

►  Responsible  Party  for  Implementation  of 
Corrective  Action 

►  Responsible  Party  for  Oversight  of  Corrective 
Action 

►  Due  Date 


QC  Review  Reponses 


BUILDING  STRONG, 


Verification,  Validation  and  Usability  Input 

WS  #34 


■  Verification 

►  Completeness  of  tasks  and  data 

■  Validation 

►  Conformance  to  Specifications 

■  Usability 

►  Achievement  of  MPCs 

►  Overall  Achievement  of  DQOs 

•  Decision  can  be  made  with  confidence 


Ident  fies  QA  Review  Inputs 


BUILDING  STRONG, 


Verification  and  Validation  Procedures: 

WS  #35 


■  Verification  and  Validation  Reviews 

►  Who  is  Responsible? 

►  What  is  Reviewed? 

►  When  (Frequency)? 

►  How  Reported? 


Lists  QA  Oversight  Review  and  Reports 


BUILDING  STRONG, 


Detailed  Validation  Procedures:  WS  #36 


■  General  Plan  for  QA 

■  How  Thresholds  are  Tested 

►  “Blind  Validation”  Seed  Plan  (Firewalled) 

►  Verification  of  the  “Threshold”  for  “Dig  List” 


BUILDING  STRONG, 


Data  Usability  Assessment:  WS  #37 


Requirements: 

■  Performed  by  “key 
members”  of  the  PDT 

■  Completed  at  conclusion 
of  each  phase  of 
investigation 

►  Delivery  Units 


Review: 

■  SPP  in  Retrospect 

■  Support  Assumptions 

►  CSM 

►  Managed  Uncertainty 

■  Representative  Data 

■  Confidence  in  Data 


BUILDING  STRONG, 


WS  #37:  Key  Personnel  and  Delivery  Unit 


Key  Personnel 

■  Project  Manager 

■  QA  Manager 

■  QA  Geophysicist 

■  QC  Geophysicist 

■  Field  Geophysicist  (Lead) 


Delivery  Unit  (DU) 

■  Each  (or  grouped)  Survey 
Units  prepared  as  a  data 
set  for  reporting. 

►  Daily? 

►  Weekly? 

►  Bulk  anomalies? 


BUILDING  STRONG, 


WS  #37:  Inputs  to  Usability 


Usability  Documents 

■  QAPP 

►  Verification  &  Validation 
Plan 

■  QASP 

■  Contract 

■  QC  Reports 

■  Corrective  Actions 

■  IVS  Memoranda 


Detection  Survey 
Validation  Report 

Site  Specific  Library 

Cued  Survey  Validation 
Report 

Prioritized  Target  “Dig” 
List 

Target  Classification 
Report 

Classification  Validation 
Report 


BUILDING  STRONG, 


WS  #37:  Process 


Step  1 . 

■  Review  Objectives  and 
Sampling  Design 

■  Is  Design  Consistent  with 
objectives? 

■  Identify  Deviations 

Step  2. 

■  Review  Verification  / 
Validation  Inputs 

■  Evaluate  Conformance  to 
MPCs  (WS  #12) 


Step  3. 

■  Document  Data  Usability 

►  Implications  of  missed  QC 
and  corrective  actions 

■  Update  CSM 

■  Draw  Conclusions 

►  Next  Phase? 

Step  4. 

■  Lessons  Learned 

■  Recommendations  for 
changes  to  DQOs  in 
future  DUs  if  needed. 


BUILDING  STRONG, 


Planning  Complete!!!! 


Develop  the 
Design 


£'  v  itKrsyy 


Assessment 
and  Usability 


Data  Quality 
Control  / 
Quality 
Assurance 


Develop  Data 
Quality 
Objectives 


Management: 
Define  PDT 
Roles  & 

Responsibilities 


BUILDING  STRONG, 


Questions? 


John  M.  Jackson 

John.M.Jackson@usace.armv.mil 

(916)  557-6614 


BUILDING  STRONG, 


zz 


BUILDING  STRONG, 


Remember  Camp  SLO? 


University  owned 

►  No  access  restrictions 

►  Cattle  grazing 

►  Geotechnical  classes 

►  Camping 

Multiple,  overlapping 
range  fans 

MRS  ~  2,500  acres 


BUILDING  STRONG, 


Former  Camp  SLO  Rl  Results 


Farmer  Camp  San  Lula  Qdlapo 
MRS  01HJZ  and  MRS  OB 
Remedial  InvesUgatlartfFeaaibmty  Study 
San  Luis  Obispo  County,  Call  Torn  la 
Figure  5-S 

MFC  and  MD  Density  MRS  OB 


Legend 
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BUILDING  STRONG^ 


Step  1-  Delineate  the  site 


Farmer  Camp  San  Lula  Qdlapo 
MRS  01^02  and  MRS  OB 
Remedial  Inves  tig  at  lartf  Feasibility  Study 
San  Luis  Obispo  County,  Call  Torn  la 
Figure  5-S 

MFC  and  MD  Density  MRS  OB 


Legend 


4  tJbnk  dtce  and  Explosives  or  Csnaerr 
z-  ML-nbore  Deads. 


- Ara-og  G soot ysl sal  Transa Os 


- DGUTrariseete 
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MedUm  Derafy  Ata  2  f14  acres'i 


Medi jit;  DeiEty  Area  3  {1 1  acres-l 
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OSLO  FU33  EdL.Tdary 


- Glare  Rale  1 


Estimated  Density  [MD  '  MEC  per  Ac  j 
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BUILDING  STRONG^ 


Depth  (meters  below  ground  surface) 


Step  2-  Develop  RAOs  for  each  (sub)MRS 


0.0 

0.2 

0.4 

0.6 

0.8 

1.0 

1.2 

1.4 


★ 

★ 

★ 
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★ 
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★ 

.... 

★ 

★ 
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Est. 

Est, 

---- 
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deepest 
recovered  UXO 


amplitude  response 
detection  limit 
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BUILDING  STRONG, 


Step  3-  Pick  one  MRS,  create  your  DQO 


7-step  Process 

■  Step  1 :  Problem  Statement 

■  Step  2:  Goals  of  the  Study 

■  Step  3:  Information  Inputs 

■  Step  4:  Boundaries 

■  Step  5:  Analytic  Approach 

■  Step  6:  Performance 
Objectives 

■  Step  7:  Detailed  Plan  to  Obtain 
Data 


Instructions 

■  Write  a  detailed  problem 
statement 

■  Provide  general 
statements  or  ideas  for 
the  other  6  steps 
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BUILDING  STRONG, 


Post  Remedy-  Scenario  1 


Pea k  rirtirclH  Siind  I  vi  Pr-ak  Nuis-e 


ipflak  Filter  Output  ol  M«iel  *  flolse 
vs  Fc-zk  Filter  Output  efNDive 
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RPMWnw 

LMKbun  1 1 

ThrtshDH 

|1.5 


&POIP  Fitter 

Coherence 
OrtKtian 
Tlvtskald  0GL2J 
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BUILDING  STRONG, 


Scenario  1-  High  density  not  completed-  data  gap 


Density  From  Dynamic  TEMTADS 


4743.6 

4487.2 

4230.8 

3974.4 

3717.9 

3461.5 

3205.1 

2948.7 

2692.3 

2435.9 

2179.5 

1923.1 

1666.7 

1410.3 

1153.8 
897.4 
641.0 
384.6 
128.2  _ 

Density 

(targets/acre) 
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BUILDING  STRONG, 
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Scenario  1-  High  density  not  completed-  data  gap 


Density  From  Dynamic  TEMTADS 


4743.6 

4487.2 

4230.8 

3974.4 

3717.9 

3461.5 

3205.1 

2948.7 

2692.3 

2435.9 

2179.5 

1923.1 

1666.7 

1410.3 

1153.8 
897.4 
641.0 
384.6 
128.2 

Density 

(targets/acre) 


High  Density  Area 
(>4,000  targets/acre 


Target  Count  Estimates: 

Inside  High  Density  area:  3,913 

Outside  High  Density  Area:  14,925 

Total:  18,838 


Questions 

■  Did  we  meet  the  RAO 
and  intent  of  the  ROD? 

■  ROD  requires  explanation 
of  significant  differenced? 

►  What  would  the  ESD  look 
like? 

■  LUCs? 

►  How  do  we  implement  on 
state  lands? 

►  How  do  we  implement  on 
private  lands? 
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BUILDING  STRONG, 


Scenario  2-  High  density  completed-  NO  data  gap 


Density  From  Dynamic  TEMTADS 


4743.6 

4487.2 

4230.8 

3974.4 

3717.9 

3461.5 

3205.1 

2948.7 

2692.3 

2435.9 

2179.5 

1923.1 

1666.7 

1410.3 

1153.8 
897.4 
641.0 
384.6 
128.2 

Density 

(targets/acre) 


High  Density  Area 
(>4,000  targets/acre 


Target  Count  Estimates: 

Inside  High  Density  area:  3,913 

Outside  High  Density  Area:  14,925 

Total:  18,838 


Questions 

■  Did  we  meet  the  RAO 
and  intent  of  the  ROD? 

■  ROD  requires  explanation 
of  significant  differenced? 

►  What  would  the  ESD  look 
like? 

■  LUCs? 

►  How  do  we  implement  on 
state  lands? 

►  How  do  we  implement  on 
private  lands? 
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BUILDING  STRONG, 


Scenario  3-  Dig  results  comparison 


3a 

3b 

Anomaly/Target  Quantities 

16,202 

Anomaly/Target  Quantities 

16,202 

Amplitude  Response 

Amplitude  Response 

Anomalies  Detected 

Anomalies  Detected 

Advanced  Detection  Analysis 

7,035 

Advanced  Detection  Analysis 

7,035 

Targets  (Potential  TOI 

Targets  (Potential  TOI 

Identified  for  Cued  Analysis) 

Identified  for  Cued  Analysis) 

Targets  Identified  for  Dig  List 

753 

Targets  identified  for  Dig  List 

753 

Can’t  Analyze 

(104) 

Can’t  Analyze 

(104) 

Potential  TOI 

(418) 

Potential  TOI 

(418) 

Calibration,  Threshold 

(231) 

Calibration,  Threshold 

(231) 

Verification  and  Validation 

Verification  and  Validation 

Digs 

Digs 

Dig  Records  in  Database 

1,456 

Dig  Records  in  Database 

1,456 

(more  than  one  item  found 

(more  than  one  item  found 

within  a  0.4  meter  radius 

within  a  0.4  meter  radius 

resulted  in  multiple  records 

resulted  in  multiple  records 

for  an  investigated  location) 

for  an  investigated  location) 

Munitions  Debris  (note  that 

(1,223) 

Munitions  Debris  (note  that 

(1,223) 

some  of  the  individual 

some  of  the  individual 

records  indicate  multiple 

records  indicate  multiple 

pieces  of  frag) 

pieces  of  frag) 

QC  or  QA  Seed 

(94) 

QC  or  QA  Seed 

(94) 

Other  Debris  (Primarily  Scrap 

(62) 

Other  Debris  (Primarily  Scrap 

(62) 

Metal) 

Metal) 

Small  Arms 

(36) 

Small  Arms 

(36) 

No  Contact 

(36) 

No  Contact 

(36) 

Other  (shared  with  adjacent 

(4) 

Other  (shared  with  adjacent 

(4) 

anomaly) 

anomaly) 

UXO 

(1) 

UXO 

(38) 

Scenario  4-  verification/validation 


4a  4b 


Anomaly/Target  Quantities 
Amplitude  Response 
Anomalies  Detected 

16,202 

Advanced  Detection  Analysis 
Targets  (Potential  TOI 
Identified  for  Cued  Analysis) 

7,035 

Targets  Identified  for  Dig  List 

553 

Can’t  Analyze 

(104) 

Potential  TOI 

(418) 

Calibration,  Threshold 
Verification  and  Validation 
Digs 

(31) 

Dig  Records  in  Database 
(more  than  one  item  found 
within  a  0.4  meter  radius 
resulted  in  multiple  records 
for  an  investigated  location) 

1,456 

Munitions  Debris  (note  that 
some  of  the  individual 
records  indicate  multiple 
pieces  of  frag) 

(1,223) 

QC  or  QA  Seed 

(14) 

Other  Debris  (Primarily  Scrap 
Metal) 

(62) 

Small  Arms 

(36) 

No  Contact 

(36) 

Other  (shared  with  adjacent 
anomaly) 

(4) 

UXO 

(1) 

Anomaly/Target  Quantities 
Amplitude  Response 
Anomalies  Detected 

16,202 

Advanced  Detection  Analysis 
Targets  (Potential  TOI 
Identified  for  Cued  Analysis) 

7,035 

Targets  Identified  for  Dig  List 

2753 

Can’t  Analyze 

(2104) 

Potential  TOI 

(418) 

Calibration,  Threshold 
Verification  and  Validation 
Digs 

(231) 

Dig  Records  in  Database 
(more  than  one  item  found 
within  a  0.4  meter  radius 
resulted  in  multiple  records 
for  an  investigated  location) 

1,456 

Munitions  Debris  (note  that 
some  of  the  individual 
records  indicate  multiple 
pieces  of  frag) 

(1,223) 

QC  or  QA  Seed 

(94) 

Other  Debris  (Primarily  Scrap 
Metal) 

(62) 

Small  Arms 

(36) 

No  Contact 

(36) 

Other  (shared  with  adjacent 
anomaly) 

(4) 

UXO 

(38) 

ACCREDITATION 


Advanced  Geophysical 
Classification  (AGC) 


US  Army  Corps 
of  Engineers. 


AGC  PROCESS  IN  REVIEW 

THE  PROCESS  IS  LINEAR 


2 


■  The  AGC  process  consists  of  three  steps: 

1.  Measure  the  response  of  a  subsurface  metal 
object  to  an  electromagnetic  field  using  an 
advanced  geophysical  sensor. 

2.  Analyze  the  measured  response  to  determine 
target  parameters  such  as  depth,  size,  aspect 
ratio,  and  wall  thickness. 


3. 


Use  these  parameters  as  inputs  to  a  classifier 
to  help  decide  whether  the  detected  item  is 
most  likely  a  munition  that  must  be 
investigated.  , _ , 

m 

US  Army  Corps 
of  Engineers 


AGC  PROCESS  (AND  DGM  IN  GENERAL)  QUALIFIES  AS 
ENVIRONMENTAL  SAMPLING 


Intergovernmental  Data  Quality  Task  Force 

An  interagency  partnership  with  a  shared  mission  and  goals 


r 
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Mission 

-  To  document  an  intergovernmental  quality  system, 
beginning  with  the  hazardous  waste  programs  (CERCLA 
and  RCRA) 

Goals 

-  To  document  an  intergovernmental  quality  system  based 
on  ANSI/ASQC  E-4 

-  To  identify  the  roles  and  responsibilities  of  EPA  and  other 
Federal  agencies  regarding  QA/QC  oversight 

-  To  develop  guidance  for  implementing  Federal 
Government-wide  requirements  and  procedures 
regarding  data  quality 


L 


US  Army  Corps 
of  Engineers 


IDQTF  PRODUCTS 


Uniform  Federal  Policy  for 
Implementing  Environmental 
Quality  Systems 

Final,  signed  by  all 
three  agencies, 

January  2003 

Uniform  Federal  Policy  for 

Quality  Assurance  Project  Plans 
(UFP-QAPP),  Manual  and 
Workbook 

Review  draft,  Fall  2003 

QA/QC  Compendium:  Minimum 
QA/QC  Activities 

Review  draft,  Fall  2003 

Roles  and  Responsibilities 
Guidance,  Appendix  to  UFP-QS 

Review  draft,  Spring 

2004 

AGC  QAPP 

Final,  March  2016 

MEC  QAPP 

Draft,  2016/2017 
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AGC-QAPP 
Revision  Number: 
Revision  Date: 


INTERGOVERNMENTAL  DATA  QUALITY  TASK  FORCE 


Uniform  Federal  Policy 
For 

Quality  Assurance  Project 
Plans 


Advanced  Geophysical  Classification  for 
Munitions  Response 

(AGC-QAPP) 

Version  1.0,  March  2016 


UNIFORM  FEDERAL 
POLICY  FOR  QUALITY 
ASSURANCE  PROJECT 
PLANS -ADVANCED 
GEOPHYSICAL 
CLASSIFICATION  FOR 
MUNITIONS  RESPONSE 
(AGC-QAPP) 


https  ://www.  epa.gov/fed 

fac/uniform-federal- 

policy-quality- 

assurance-project- 

plans-template- 

advanced-geophysical 


US  Army  Corps 
of  Engineers 


COMMITMENT  TO  IDQTF  PRODUCT 


r 
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•  Each  agency  will  decide  how  best  to  implement  products. 

•  Each  agency  will  be  responsible  for  its  own  oversight. 

•  IDQTF  will  have  a  continuing  role  to  address  issues  and 
ensure  improvement. 


L 


US  Army  Corps 
of  Engineers 
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AGC  IMPLEMENTATION  IN  THE  DOD 
DAGCAP  IS  THE  NEW  NORMAL 


L 


US  Army  Corps 
of  Engineers 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
defense  pentagon 

WASH  INGTON .  DC  £030 1  -lAtia 


APR  1  i  im 

EMERCY. 

:HSil  ALI_n.TIE«'.MK. 

>«□  EMVIROhMENF 

MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  THE  ARMY  (JNSTA 1 .1  .AT10NS, 
ENERGY  AND  ENVIRONMENT) 

ASSISTANT  SECRETARY  OF  THE  NAVY  (ENERGY. 

INSTALLATIONS  AND  ENVIRONMENT) 

ASSISTANT  SECRETARY  OF  THE  AIR  FORCE 
(INSTALLATIONS,  ENVIRONMENT  AND  ENERGY) 

DIRECTOR,  DEFENSE  LOGTSTTCS  AGENCY  (DSS-E) 

SI.  EJECT:  Department.  of  Defense  Advanced  Geophysical  Glassification  Accreditation  Program 

The  Department  of  Defense  (DuD)  developed  die  mu  n  iT.i  \t\: r-  response  advanced  goo  physical 
classifiemkm  process  [hereinafter  referred  to  as  advanced  classificaLion)  bi  improve  the  efficiency 
of  cleaning  up  munitions  sr.d  to  focus  resources  on  potential  explosives  safely  risks  at  munitions 
response  ft£e$  (MRSs).  To  ensure  quality  dalu,  my  office  has  estabijslied  the  I >ID  Advanced 
Geophysical  Classification:  Accreditation  Program  (DACiCAPj  [o  accredit  organizations  Lhat  use 
advanced  elassi  kculinn  at  MRSs.  The  DACCA  I1  i*  modeled  sifter  the  laboratory  accreditation 
program. 


TL:c  DAC.R.AP  provides  a  unified  program  for  organizations  perfenning  advanced 
classification  to  demonstrate  competency  and  document  eonlermance  to  minimum  quality  systems 
reiii.irncmer.rs  based  on  the  International  Organization  for  Standardization  and  the  Itnermiilionu] 
Flwtrotwchnicai  Commission  standards.  DoD  ensures  quality  control  measures  a rc  in  pi  nee  to 
satisfy  both  DeD  project  managers  and  regulators  by  Ibe  accreditation  process.  AtLrcdil.iii.ior  ls 
achieved  by:  (1 )  assessment  of  the  organization^  qiiiitiLy  system;  and  (2)  a  successful  demon*  iraEiou 
Of  capabilities  performed  at  Ihe  Aberdeen  Piovlng  Ground  DAGCAF  test  site.  The  attachment 
outlines  the  accreditation  process  and  also  includes  frequently  asked  questions.  The  DaGCAP 
webpugc  on  die  MiUtaiy  Muni  dens  Response  Program  page,  h  1 1  p :  /.■'  wvvvv ,  d  e.n  i  x .  os  d ,  nil  I A  iitti  rp ,  will 
inaiiuaiu  the  latest  documentation  and  stakeholder  iofon nation. 


OrgariL^Liums  may  begin  the  DAG  CAP  accreditation  process  in  the  second  quartet  of 
calendar  year  2016.  The  DoD  Components  shall  begin  using  accredited  organizations  ol:  their 
MRSs  beginning  in  calendar  year  2017. 


My  points  of  contact  tor  the  DAGCAl1  are  D:.  Juniun  Adcl&om  DoD  Environmental  Di-l-i 
Quality  Workgroup  Chair,  at  jordau.addson@nav>r.mi];  and  Ms.  Deborah  Morcfield, 
ODASD(ESOH).  at  dc borali.a.Tnorefi  eM.civ.jSJntai  L.  tu i  L. 


DepuLy  Assisildni  S  e  cretruy  of  Ds  fen  *  e  (R  us  i  n  g) 
Performing  due  duties  of  the  Assistant  Secretary  of  Defence 
(Energy,  Installations smr!  Environment  1 


Attachment: 
As  stated 


DoD  ADVANCED 

GEOPHYSICAL 

CLASSIFICATION 

ACCREDITATION 

PROGRAM 

(DAGCAP) 


Memo  signed  April  11 , 
2016,  established 
DAGCAP  to  accredit 
organizations  that  use 
AGC  at  MRSs. 


US  Army  Corps 
of  Engineers 


humth 
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DAGCAP  OVERVIEW 

■  Modeled  after  DoD  Environmental  Laboratory 
Accreditation  Program  (ELAP) 

■  Third-party  Accreditation  Bodies  (ABs) 
conduct  assessments  to  Quality  Systems 
Requirements  (QSR) 

■  Applies  to  all  testing  organizations  regardless 
of  size  or  volume  of  business 

■  Applies  to  use  of  AGC  at  all  MRSs 


US  Army  Corps 
of  Engineers 


A  STATE’S  PERSPECTIVE  ON 
ADVANCED  GEOPHYSICAL 
CLASSIFICATION  AND  THE 
EVOLUTION  OF  MMRP 

COPIED  FROM: 

2016  NATIONAL  FUDS  FORUM 
AUGUST  2016 

PRESENTED  BY:  TRAC  IE  WHITE  &  JEFF  SWANSON 
COLORADO  DEPARTMENT  OF  PUBLIC  HEALTH  &  ENVIRONMENT 


H 

hjumuti 


US  Army  Corps 
of  Engineers 


BIG  PICTURE  -  KEY  IDEAS 
ADVANCED  SENSORS  &  GEOPHYSICAL 
CLASSIFICATION 

Technology  Acceptance  &  Adoption 

-  It  works,  really  good! 

-  Not  a  “silver  bullet”,  not  appropriate  everywhere 

Quality  Systems  is  the  key  to  success 

-  Accreditation  is  a  game  changer 

-  QAPP  provides  the  quality  framework 

-  Sustain  with  policy,  training,  and  implementation 

Implementation  &  Oversight 

-  Focus  on  decision  points 

-  Classification  is  hands-on  technology 

-  Requires  active  stakeholder  participation 

US  Army  Corps 
of  Engineers, 
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CLASSIFICATION  TECHNOLOGY 


Classification  is  a  disruptive  technology 

-  Changing  how  munitions  cleanup  is  conducted... 

-  Driving  several  important  secondary  changes... 


Major  changes 

-  Contracting 

-  Planning  &  Design 

-  Field  Implementation 

-  Quality  Systems 

-  Corps  Oversight  Role 

-  State  Oversight  Role 

Touches  all  Stakeholders 


n 


Traditional  Change  Model 


?>  20 10  by  Velaction  Continuous  Improvement,  LLC. 
All  rights  reserved . 


www.Velaction.com 
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US  Army  Corps 
of  Engineers, 
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EVOLUTION  OF  MUNITIONS 
CLEANUP 


Technology  Gen  3 

-  Advanced  sensors 

-  Classification  decisions 

Work  Force 

-  Smaller  work  force 

-  Multiple  field  mobs 

-  Reduced  digging  cost 

Cost  &  Complexity 

-  Significant  cost  savings 

-  Increased  complexity  of 
decision  making 

-  New  oversight  &  quality 
requirements 


I  I 

Mag  &  Dig  |  Digital  Mapping 


COLORADO’S  EXPERIENCE 


Former  Lowry  Bombing  &  Gunnery  Range 
Geophysical  Classification  Demonstration 

Project  Site 

-  57mm  Recoilless  Rifle  Range  (GM  MRS) 

-  57mm,  37mm  projectiles  (20mm?) 

Scope  of  Demo 

-  Leave  it  in  the  ground  -  No  1 00%  ground  truth 

-  Start-to-finish  demonstration  of  complete  MEC 
removal  project 

•  Contracting,  QAPP  implementation 

•  USACE  &  State  QA  Oversight 

Technology 

-  EM61-MKII  &  MetalMapper  detection  surveys 

-  MetalMapper  dynamic  classification  survey 

-  MetalMapper  cued  classification  survey 

Objective 

-  Safe  for  future  recreation  use  and  development  next 
to  new  major  reservoir 


LESSONS  LEARNED 

LOWRY  CLASSIFICATION  DEMONSTRATION 

Communication,  communication,  communication 
Remedial  Design  &  Planning 

-  QAPP  effective  tool  as  project  work  plan 

-  Data  quality  objectives  must  be  munitions  specific 

-  All  QAPP  Worksheets  are  important 

-  SOPs  must  be  detailed  and  implementable 
Field  Implementation 

-  Don’t  “Go  rogue”  -  Field  teams  must  follow  QAPP  & 
SOPs 

-  Rigorous  QC  program  -  seeding  and  i 

-  Full  root-cause  analysis  (5  Why’s) 


nspections 

V\ 

. 

r  ^ 

KEYS  TO  REGULATORY  ACCEPTANCE 

REMEDIAL  DESIGN 

Begin  at  the  beginning,  before  contracting 

-  Agreement  on  fundamental  design  and  use  of 
classification 

Establish  clear  RAOs  for  each  munitions  item 

-  Identify  all  munitions  of  concern  and  depths 

-  Must  be  able  to  detect  before  you  can  classify 

Establish  verification  and  validation  requirements 

-  IVS,  QA  seeds,  Library  Items,  Verification  Digs 


US  Army  Corps 
of  Engineers 


UMilil'U 
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r  ^ 

KEYS  TO  REGULATORY  ACCEPTANCE 

CLASSIFIER  DECISION  POINTS 


No  “Black  Box”  analysis  or  decisions 

Transparent  decisions  -  Detection  &  Classification 

-  Understand  all  decision  points 

-  Well  documented  decision  trees 

-  Establish  decision  thresholds,  criteria,  and  standards 
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Verification  and  Validation  Strategy 

-  Specifications  for  data  quality  and  monitoring 

-  Classifier  models  and  decision  thresholds 

-  Final  project  results 


“Make  the  right  decisions,  for  the  right  reasons.  ”  /wSmlfe 
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KEYS  TO  IMPLEMENTATION 

QUALITY  ASSURANCE  OVERSIGHT 

Demand  rigorous  remedial  design 

-  Clearly  documented  in  QAPP 

-  Detailed  and  implementable  SOPs 
Disciplined  field  implementation 

-  Strict  adherence  to  QAPP  and  SOPs 

-  Not  acceptable  to  “get  to  the  field,  then  figure  it  out” 

QA  as  QA 

-  Expect  QC  program  to  find  and  fix  issues 

-  No  excuses  at  QA  level  -  failing  QA  is  a  big  deal 

-  Root  cause  analysis  and  corrective  actions  (5 
Why’s) 
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KEYS  TO  IMPLEMENTATION 

STATE  OVERSIGHT  FOCUS 

Remedial  design 

-  Verify  approach  for  detection  and  classification 

-  Confirm  munitions-of-interest  and  depths-of-interest 

-  Ensure  QAPP  fully  documents  project 

Decision  point  verification 

-  Verify  data  quality  objectives  are  met 

-  Validate  detection  &  classification  decisions 

Quality  systems  implementation 

-  Verify  full  implementation  of  QAPP  and  QC/QA 

-  Independent  QA  and  validation 

-  Rigorous  root-cause  analysis  and  corrective  actions  _ 
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IMPLEMENTATION  CHALLENGES 
STATE  OVERSIGHT 


Project  Manager  training 

-  Technical  fundamentals,  design,  quality  systems  and  tools 

-  Sharing  of  Lessons  Learned  and  practical  experience 
Independent  Technical  Support 

-  Access  to  independent  technical  experts 

-  Experts  to  validate  design  and  classifier  models 

-  Independent  data  and  analysis  verification 
Technical  Guidance  on  State  Oversight  Plans 

-  Document  available  tools  and  expectations 

-  Worksheet  companion  to  QAPP  template 
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QUESTIONS? 
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QAPP  NINJA 
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COMMUNICATING  RISK 
MANAGEMENT 


Advanced  Geophysical 
Classification  (AGC) 
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AGENDA 


Basic  QAPP 
Assessing  End  States 
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WHY  BASIC  QAPP? 

WHEN  ACCEPTING  DATA  &  CLOSING 
OUT  PROJECT 

Critical  Worksheets  according  to  Andy 

-  WS10:  Conceptual  Site  Model 

-  WS11 :  Data  Quality  Objectives 

-  WS12:  Measurement  Performance  Criteria 

-  WS22:  Measurement  Quality  Objectives 


Because  the  QAPP  is  our  best  guarantee  we’re 
using  the  right  data  for  all  our  decisions 
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BASIC  QAPP 

WHEN  ACCEPTING  DATA  &  CLOSING  OUT 
PROJECT 

WS10:  Conceptual  Site  Model 

-  Foundation  of  entire  project 

-  Explains  why  we  planned  what  we  planned 


Study  Boundary 


VSP  Analysis  (Kriging)  usii  u 
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BASIC  QAPP 

WHEN  ACCEPTING  DATA  &  CLOSING  OUT  PROJECT 

WS11 :  Data  Quality  Objectives 
-  GCMR  QAPP  template  is  comprehensive 

•  Only  needs  minor  editing  for  project  specific  thresholds 

•  Explains  solutions  to  relatively  simple  problem: 

Which  anomalies  are  TOI,  which  are  not 


US  Army  Corps 
of  Engineers 


r 


WS  11  FROM  THE  TEMPLATE 
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Step  1 :  State  the  Problem.  Define  the  problem  that  necessitates  the  study. 
Examine  budget  and  schedule  issues. 

Site-specific  problem  statement:  (Example)  Previous  investigations  (list)  have 
indicated  that  MEC  in  the  form  of  DMM  and  UXO  including  (x,  y,  and  z)  are 

present  at  site _ ,  resulting  from  its  use  between  (years) 

_ and _ as  a  (describe  the  type  of  facility  and  its  uses).  As 

shown  in  the  CSM  these  materials  present  an  unacceptable  risk  from  explosive 
hazards  to  (describe  current  receptors  and  potential  future  receptors  based  on 
anticipated  land  use.) 

Advanced  geophysical  classification  uses  advanced  sensors  and  geophysical 
classifiers  to  estimate  physical  properties  of  the  item  (e.g.,  depth,  size,  aspect 
ratio,  wall  thickness,  symmetry)  and  determine  whether  the  item  is  a  TOI  (i.e., 
highly  likely  to  be  MEC)  or  non-TOI  (i.e.,  highly  unlikely  to  be  MEC).  Using  this 
information  in  a  structured  decision-making  process,  project  teams  will  be  able  to 
make  informed  decisions  about  whether  an  item  should  be  excavated  or  can  be 
left  in  place. 

US 
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BASIC  QAPP 

WHEN  ACCEPTING  DATA  &  CLOSING  OUT  PROJECT 

WS12:  Measurement  Performance  Criteria 
-  The  single  most  important  Worksheet 

•  Because  if  we  meet  all  these  requirements,  we’re  DONE! 

•  GCMR  QAPP  template  is  comprehensive 

-  Edits  usually  not  needed 
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BASIC  QAPP 

WHEN  ACCEPTING  DATA  &  CLOSING  OUT  PROJECT 

WS22:  Measurement  Quality  Objectives 

-  Second  most  important  worksheet 

-  Tells  us  the  data  we’re  using  is  good 

-  GCMR  QAPP  template  is  comprehensive 

•  Minor  edits  only  for  project  specific  needs 
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ASSESSING  END  STATES 
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AGENDA 


First  Up:  Need  to  make  you  smart  on  what  you’re  starting 
out  with 

Then:  Dive  in  to  implementing  and  defending  our  exit  plan 
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CONTEXT 


How  We  Got  Here... 

RI->FS->PP/DD->RD->RA  QAPP  followed  by  field  work 

RI->Baseline  Risk->Have  An  Unacceptable  Risk  Scenario 
This  means,  Per  40  CFR  Part  300.430(e)(i),  the  Lead  Agency 
established  remedial  action  objectives  (RAOs)  that 
specify: 

-  contaminants  and  media  of  concern 

-  potential  exposure  pathways,  and 

-  remediation  goals” 


11 


US  Army  Corps 
of  Engineers 


GENERAL  RESPONSE  ACTIONS 

The  Three  General  Response  Actions  in 
our  GRA  Toolbox 

Modify  Behavior 

Restrict  /  Preclude  Remove  the  Source 
Access 
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GENERAL  RESPONSE  ACTIONS 


The  four  General  Response  Actions  in  our 
GRA  Toolbox 


Physical 

Removal 
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RAO  IN  PLAIN  ENGLISH 


HTRW:  “Meet  this  standard”  (i.e.  30ppm  Pb  in  soil) 

MMRP:  When  we  can  show  either: 

1 .  People  aren’t  likely  to  encounter  UXO,  or 

2.  People  know  what  NOT  to  do  if  they  encounter  a 
UXO, 

3.  The  consequences  are  not  severe,  or 

4.  A  combination  of  the  above 
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RISK  CALCULATIONS 


Andy’s  Auto  Insurance 

Who’s  ahead,  Andy  or  his  ^ 

insurance  company?  % 

Event 

Value 

Sound  System 
theft  2010 

$1,500 

Sound  System 
theft  201 2 

$1,000 

Crazed  Soccer 
Mom  driving  a 
minivan  talking  on 
the  phone 

$6,700  1 

Total  Claims 
Payouts 

$9,200  1 

My  approximate 
lifetime  premiums 
paid  to  date 

~$350/6mo  for  1 2  | 

years  =  $8,400  i 

-  probability  of  having  an  accident 

Hours  on  the  road,  your  experience, 
your  driving  history 


WE  ARE  TALKING  PROBABILITIES 


What’s  the  probability  of  rolling  a  6? 

One  in  six,  or  about  1 6% 

What’s  the  probability  of  rolling  two  sixes? 
one  in  six?  (~16%) 
two  in  twelve?  (~16%) 
one  in  six  +  one  in  six?  (~32%) 
one  in  six  X  one  in  six?  (~3%) 
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RISK  CALCULATIONS 


Risk  is  essentially  the  multiplication  of  several 
probabilities 
Example:  Lead  In  Soil 

-  probability  of  exposure 

(being  present  where  the  contamination  is) 

-  probability  of  intake 

(something  happens  that  results  in  ingestion) 

-  probability  of  bioloading 

(probability  your  body  retains  the  contaminant) 

-  probability  of  adverse  health  effect 

(probability  that  the  retention  ultimately  leads  to  a  health 
effect) 
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SOMETHING  BAD  HAPPENS  WHEN: 


S  UXO  or  DMM  is  at  a  location 
■s  Someone  imparts  energy  to  the  item 
s  The  item  functions 

s  Energy  from  the  detonation  injures  that  someone 
Consider  Reality: 
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UXO  are  generally  rare 

Just  because  it’s  there  does  not  mean  someone  finds  it 
If  it  is  found,  it’s  not  always  picked-up 


If  it  is  picked  up,  it  doesn’t  automatically  detonate 

If  it  does  detonate,  injury  is  proportional  to  energy 
release  li!31 
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RISK  MANAGEMENT 


Lot  Of  Overlap  »  Multiplying  Large  Probabilities 


=Unacceptable  Risk 


LET’S  LOOK  AT:  PHYSICAL  REMOVAL 


Little  To  No  Overlap  »  Multiplying  small  probabilities 

=  Acceptable  End  State 


US  Army  Corps 
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LET’S  LOOK  AT:  EDUCATION 


These 
stay  the 
same! 


Little  To  No  Overlap  ~  Multiplying  small  probabilities 

=  Acceptable  End  State 
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EXAMPLE 


■  Potential  Acceptable  End-States 

>  CSM: 
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Documented  Use  As  Target  Areas 
Confirmed  Use  as  HE  Training 
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Figure  4.1 

Right  of  Entry 
Formerly  Used  Defense  Site 
Spencer  Artillery  Range 
FUDS  Project  #  G04TN0 17801 

Spencer,  Tennessee 

Legend 

I  I  Bald  Knob  Investigation  Area 

□  Jake's  Mountain  Investigation  Area 

□  Residential  Investigation  Areas 
Trail  of  Tears 

Potential  Area  of  Interest 
|2^j|  FUDS  Boundary 

Right  of  Entry  -  for  parcels  within 
investigation  area 

|  No  Response  (No  access) 

|  Access  Denied 
□□  Yes  (Access  Granted) 

Notes: 

Although  status  is  indicated  for  entire  parcel  only  portions 
of  parcels  within  investigation  area  boundanes  required  access. 
GPO  =  Geophysical  Proveout 


Interaction  expected  during  land  use 


Current  and 
planned  land 
use: 

1  -  Residential 
2-  Agricultural 


EXAMPLE 

■  Potential  Acceptable  End-States 


>  100%  coverage  &  High  Confidence  UXO  recovery  ->Unlikely 
Encounter 


>  Potential  for  UU/UE 

>  5-year  reviews  if  no  UU/UE 

>  Are  LUCs  or  ICs  needed? 
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I  I  seed  interval 

-  UXO 

-  inert 

★  t  ★  deepest  recovery 
— - detection  depth 
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Baseline  Risk 
Assessment 


Is  an 
Unacceptable 
Hazard 
.Identified?. 


<NO: 
Response 
Complete 


<Yes 

Define 
Acceptable 
End  States 


You  Are  Here 


Response 

Complete 


Acceptable 
End  States 


1)  Feasibility  Study 
■2)  Proposed  Plan  &  DD 

3)  RD/RA  &Collect  Remedy  Data 

4)  Post  Remedy  Assessment 


Yes: 

UU/UE, 

5YRs  are 
required 

Response 

Complete 


Unacceptable 
End  State 


EXAMPLE 


■  Potential  Acceptable  End-States 

>  100%  coverage  &  High  Confidence  UXO  recovery  ->Unlikely 
Encounter 

>  Potential  for  UU/UE 

>  5-year  reviews  if  no  UU/UE 

>  Are  LUCs  or  ICs  needed?  >No.  LUCs  or  ICs  don’t  provide  any  added  benefit 

Already  at  Unlikely  Encounter 


>  100%  coverage,  Moderate  Confidence  UXO  recovery->Seldom 

>  Are  LUCs  or  ICs  needed? 


I  I  seed  interval 

-  UXO 

-  inert 

★ t  ★  deepest  recovery 
—  —  —  detection  depth 


Baseline  Risk 
Assessment 


Is  an 
Unacceptable 
Hazard 
.Identified?. 


<NO: 
Response 
Complete 


Yes 


<  Define 
Acceptable 
End  States 


1)  Feasibility  Study 
►2)  Proposed  Plan  &  DD 

3)  RD/RA  &Collect  Remedy  Data 

4)  Post  Remedy  Assessment 


You  Are  Here 


Response 

Complete 


Acceptable 
End  States 


Unacceptable 
End  State 


EXAMPLE 


■  Potential  Acceptable  End-States 

>  100%  coverage  &  High  Confidence  UXO  recovery  ->Unlikely 
Encounter 

>  Potential  for  UU/UE 

>  5-year  reviews  if  no  UU/UE 

>  Are  LUCs  or  ICs  needed?  >No.  LUCs  or  ICs  don’t  provide  any  added  benefit 

Already  at  Unlikely  Encounter 


>  100%  coverage,  Moderate  Confidence  UXO  recovery->Seldom 
>  Are  LUCs  or  ICs  needed?  >Yes  ^Unlikely 

•  LUCs/ICs  achieve  Unlikely  to  impart  |nei|gy  when  encountered 

•  Program  LUCs/ICs  costs  indefinitely! 

•  5-year  reviews 
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EXAMPLE 

■  Potential  Acceptable  End-States  (con.t) 

>  Large  contiguous  area  with  100%  (or  reduced  coverage  under 
infrastructure)  coverage  and  No  UXO  or  DMM  or  other  native  TOI 
recovered 

>  Indicates  buffer/safety  zones  between/around  targets 

>  Potential  for  UU/UE?  >Yes 

>  5-year  reviews  if  no  UU/UE 

>  LUCs  or  ICs?  Don’t  provide  any  added  benefit  ->  already  at  Unlikely  Encounter 

>  100%  (or  reduced  coverage  under  infrastructure)  coverage,  High 
confidence  intact  UXO  recovered,  moderate  confidence  small, 
modest  injury  potential  fuze  components  recovered 

>  High  confidence  intact  UXO  recovered  ->  achieves  Unlikely  Encounter  for  those 
items 

>  Moderate  confidence  fuzes  recovered  ->  achieves  Seldom  Encounter  for  those 
items 

>  LUCs  or  ICs?  Not  |jke|y  Both  cases  Acceptable  j||jj 

US  Army  Corps 
of  Engineers, 
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SUMMARY 


We  Achieve  one  or  more  GRAs  through 

-  Physical  Removal 

-  LUCs 

-  Combination 

Risk  is  the  product  of  probabilities  (in  our  current 
formulation) 

p(presence)  x  p(encounter)  x  p(detonation)  x  p(injury) 

LUCs  need  to  increase  effectiveness  of  the  remedy,  AND 
the  remedy  effectiveness  needs  the  bump! 

More  than  one  End  State  definition  is  likely  throughout  an 
MRS 
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VISUAL  SAMPLE  PLAN 


Advanced  Geophysical 
Classification  (AGC) 


US  Army  Corps 
of  Engineers. 


Things  you  need  to  know  about 
Visual  Sample  Plan 

The  VSP  Sparse  Transect  Sampling  Tools  have 
been  completed  and  demonstrated. 


Statistical  Transect  Design 

►  Quantify  the  probability  of 
traversing  and  detecting  a 
target  area  with  a  specified 
transect  spacing. 

Detection  Performance  (6CH3  ft  Radius  ,  1m  wide  transects  f 

|  _  9kg  Density  =  AWajcr*  (Avnrngti  Target  DarsilyJ 


Sparse  Transect  Analysis 

►  Sparse  transect  density 
analysis  tools  developed 
specifically  for  DoD  surveys. 

►  Geostatistical  Modeling  and 
mapping. 


VSP 


Purpose  is\  to  find  area(s)  with  elevated  anomaly  densities. 
Purpose  is  not,  to  delineate  extent  of  UXO,  or  even  to  find  UXO 


Design 

CSM  accuracy  is  not  critical 

-  VSP  will  tell  you  how  to  find  the 
“area”  you  define 

-  Can  bias  towards  conservative 
assumptions 

-  Okay  if  you  get  it  wrong 
Can  be  step-wise  process 

-  Look  for  larger  areas  first 

-  Look  for  smaller  areas  if  needed 


Analysis 

Does  not  incorporate  design  CSM 

Will  tell  you  what  you  found 

Can  run  multiple  scenarios  to  test 
various  assumptions 


US  Army  Corps 
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UNDERLYING  PRINCIPLE 


The  “thing”  we’re  looking  for 
has  more  anomalies  per 
acre  than  the  area 
surrounding  it 
We  need  at  least  one 
transect  through  that 
“thing”  (sometimes  more, 
VSP’s  planning  module 
will  tell  us) 
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SIMPLIFIED  EXAMPLE 


L 


SIMPLIFIED  EXAMPLE 
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SIMPLIFIED  EXAMPLE 
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Scale: 


500m 

♦  » 
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SIMPLIFIED  EXAMPLE 

What  Can  We  Deduce? 
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VSP  BASICS 


Model: 

60mm  M49A3  fragmentation  distribution 
(1500  rounds) 


Detection  Confidence 
Purple  >25  anomalies/acre 
Blue  >15  anomalies/acre 
Yellow  >5  anomalies/acre 


Initial  R&D  Problem  was  to 
i  find  this 

Design  Phase  uses  Monte 
Carlo  simulations  to 
estimate  probabilities  of 
traversing  area  & 
detecting  anomalies 
Analysis  Phase  just  looks  at 
the  data 

-  Geostatistics, _ , 

ESI 

US  Army  Corps 
of  Engineers 


GENERAL  CONCEPTS 


If:  Many  munitions  types 
&  Heavy  use 


■  Then:  Start  looking  for 
Largest  Area  First 


Known: 

220mm 

155mm 

105mm 

75mm 

37mm 


his- to  nr:  F&ange-s  and  Fh-ntogi  aph  t 
AnaFlftli  f-e#tur«- 
Formally  Uj.nl  Dtaramo  Slfea 
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GENERAL  CONCEPTS 


If:  Many  munitions  types 
&  Heavy  use 


■  Then:  Start  looking  for 
Largest  First 


Known: 

220mm 

155mm 

105mm 

75mm 

37mm 


Kriged  Estimate  s 

(Anomalies/Acre) 
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GENERAL  CONCEPTS 


If:  Don’t  find  what  you  "  Then:  Do  more 

expected 


GENERAL  CONCEPTS 


If:  Don’t  find  what  you  "  Then:  Do  more 

expected 
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GENERAL  CONCEPTS 


If:  Don’t  find  what  you  "  Then:  Do  more 

expected 


GENERAL  CONCEPTS 


If:  Don’t  find  what  you  "  Then:  Do  more 

expected 
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YOU  GET  TO  DEFINE  “TARGET  AREA” 


Any  scenario  where  there  is  a  contrast 
between  “background”  anomaly  densities 
and  densities  in  the  “area”  you  are  trying  to 
find 

-  Typical  practice  target  (normal  distribution) 

-  Open  Detonation  area  (normal  distribution) 

-  Maneuver  area  (uniform  distribution  model) 

-  Could  even  be  a  disposal  pit  (but  the 
geophysical  detection  definition  will  be  different) 


Original  R&D 
Problem 
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Model  showing  all  anomalies 
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MRS  Boundary 
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|Example:  Contrast 
Iniform  Distribution 

250  ACRE  MRS 

SUSPECTED 

5AC 

MANEUVER 

AREA 

BACKGROUND 

ANOMALY 

DENSITY: 

10  BKG/ACRE 

(UNIFORM 

DISTRIBUTION) 

MANEUVER 

AREA 

DENSITY: 

1 0  /(ACRE 

(unIHixl 

DISTRIBUTION) 


Model  showing  all  anomalies 

1200 
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Example:  Contrast 
Uniform  Distribution 

250  ACRE  MRS 

SUSPECTED 

5AC 

MANEUVER 

AREA 


BACKGROUND 

ANOMALY 

DENSITY: 

10  BKG/ACRE 

(UNIFORM 

DISTRIBUTION) 


MANEUVER 

AREA 

DENSITY: 

“She  a 

DISTRIBUTION) 


Model  showing  all  anomalies 

1200 


Example:  Contrast 

ugSPASftEtMRS 

SUSPECTED 
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CASE  STUDY 

CASTNER  RANGE,  FT  BLISS 

Size  (7,000  acres) 

Three  Factors  to  define 
target  shape 

-  Where  rounds  land 

-  Where  the  frag  goes 

-  Firing  directions 


MRS  LOCATION 
Fort  Bliss,  Texa^ 


El  Paso  Museum 
of  Archaeology  v 


/  ■  Bonder  Patrol 

Museum 


m  1 


VSP  EXERCISE 


Assumed  Munition:  2.36” 
Rocket 

-  Semi-major  Axis: 

6m  +  38m  =  44 

-  Semi-minor  Axis: 

2m  +  38m  =  40 

Random  Target  Orientation 
VSP  Calculated  Transect 
Spacing  =  57m 
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DGM  ANOMALY  DENSITIES: 
VARIABLE  MV  THRESHOLD 


Set  anomaly  threshold  low  (4m V)  to  include  frag  from  HE 
rounds.  Trying  to  delineate  target  areas. 


Legend 


Anomalies  gbove  4m V 


Lyqund 

Jtnom [HP  ferfl 


Anomalies  above  GmV 


Anomalies  above  EmV 
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WHERE  ARE 
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QUIZ  TIME! 
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QUIZ  #1.  WHAT  DO  I  NEED  AT  THE  END  OF  THE  PROJECT? 
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QUIZ  #2.  FORMER  CAMP  SAN  LUIS  OBISPO  -  7ACRES  3 

REMEDIAL  RESPONSE 


L 


I  -firmer  E^ump  Lum  Dbmpn 

M  RtS  01. 02  And  OS 
RcrrreEflal  Invo-s-tlg-atlon  Fc-s-E-ibilKy  Study 

.1{»n  Lum  'Jb.hF-n  'L-nunty,  ITjilrtorrHH 

FiSw»-*  E-1 

Sab+i  Lo-cwtmn 


Lfflind 

I  ]  MBOMH 

I  |  r.i::wii 

| _ Sjt  -iij  C M^po  lITj1  LI'i  I: 

I  1  OSLO  FUOS  EfrJUJ* 

- H  Jr,1,  jv 


Kr>>frF:-' 

L'Ji  Ctw?.  3#'  Ln-Otrwa.  C#  *t-TW. 
•^13F?1td  b*  U  5.  :->.-±HilH  H>  fTl**' 

kur-JE-..  K  -|ftl  it  ■  »ll 


n 

•  lir.Tj  - 

V  x 


Firp  rl  I  ■  ■■.Jirq  , ,  . 

ypi 


h.: r\ -  :-.  -.■  ■  h -iha  *■«  ■»!-! 


'  3-  'T:  l  I 


LLLi 


".  i 


US  fttmy  C-; 

f  *  IJ  L'ji  ^JFjLHiL  MLri 


Q  srps  :>t  Engineers 


Prpp-'W'!  «*Ii  £  *1-rw b flTTW. I : ■  injrm ftr^.r rjn HTH  ■  [»V  >t  '  -hi '  JWiJ.*-:*  -"_*±7r  ™j  VI4J3HI 


US  Army  Corps 
of  Engineers 


E 

1 

HH-lil'IH 

Aftrognd.’IGNl,  l,GP  swisstopp; and  ft 


QUIZ  #2.  FORMER  CAMP  SAN  LUIS  OBISPO  -  7ACRES 

LOCATION  MAP 

LAND  USE:  Education  &  Cattle  Grazing 


...... . .  .  . . 


A  X 


B 
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QUIZ  #2.  FORMER  CAMP  SAN  LUIS  OBISPO  -  7ACRES 

SCOPING  INFORMATION 


LAND  USE:  Education  &  Cattle  Grazing 


Very  lii^li  deiiisiiy 
regLmi  hiLiitdar^ 


^liith  Btumdnry 


VSP  Ann  lysis  (Kri^ing)  Lining  TEMTADS 
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Kriged  Anomaly  Density 


d 


Hbh 


rdU 


|  m$  iflOH  Vi  ii 

AncimjIlPS  t  AtTt 


Anticipatel  4,000 
Anomalies 
Estimate  $1 .4M 
w/AGC 
Estimate  $3M 
w/out  AGC 
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QUIZ  #2.  FORMER  CAMP  SAN  LUIS  OBISPO  -  7ACRES 

...MORE  SCOPING  INFORMATION 


Remedial  Investigation 

Site-wide  Vertical  CSM 


Legend 
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Deepest 
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QC/GA  depth  range 
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Peak  Filter  Output  of  Model  +  Noise 
vs  Peak  Filter  Output  of  Noise 
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QUIZ  #2.  FORMER  CAMP  SAN  LUIS  OBISPO  -  7ACRES 

REMEDIAL  RESPONSE  RESULTS 


QUIZ  #2.  FORMER  CAMP  SAN  LUIS  OBISPO  -  7ACRES 

REMEDIAL  RESPONSE  RESULTS 


□  Delineation 

□  UU/UE 

□  Deed  Restriction 

□  Con^gpi'  |n 

Su^ort^k 

□  Signage  oM 
Educati^^ 

□  Other  Me 
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LAND  USE:  Education  &  Cattle  Grazing 
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If  Deepest 
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Treatability  Study 
QC/QA  depth  range 


Amplitude 
Response  Detection 
Performance 
(any  orientation) 

Advanced 
Detection 
Performance 
(any  orientation) 
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QUIZ  #3.  FORMER  CAMP  WHEELER 

TCRA  Results 
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□  Sign^e  or 

□  Other  LUC 
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QUIZ  #4.  FORT  ORD 


“California  Characterization” 
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QUIZ  #4.  FORT  ORD 


If  This  Is  The  Plan 
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QUIZ  #5.  FORT  ORD 


If  This  Is  The  Plan  &  What  We  Find 


□  Delineation 

□  UU/UE 

□  Deed 
Restriction 

□  Construction 
Support 

□  Signage  or 
Education 

□  Other  LUC 
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PEOPLE  WILL  LIKE,  AND  COME  TO  EXPECT 
HIGH  CONFIDENCE  DECISIONS 


Suggested  Path  Forward:  Surface  clearance  &  vegetation 
removal  tasks  become  Site  Preparation  task:  Clear  the 
surface  of  all  MEC,  reduce  vegetation  to  meet  AGC  needs, 
and  reduce  anomaly  densities  to  less  than  3,500/acre. 


r 
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RECOMMENDATIONS  FOR  MMRP  FS,  PP,  &  DD 


Consideration  of  LUCs 

If  RAO  can  be  achieved  via 
physical  removal,  no  LUCs 
necessary. 

If  physical  removal  CSM  =  Rl 
CSM,  remedy  is  achieved. 

If  physical  removal  CSM  *  Rl 
CSM,  and  RAO  not  achieved, 
ESD  may  be  required  if  LUCs 
achieve  RAO. 


Consideration  of  UU/UE 

Separate  from  assessment  of 
RAO  Achievement. 

5  YrR  required  if  remedy 
implemented  that  does  not 
achieve  UU/UE 
Document  in  DD 

Based  on  comparing  pre¬ 
clean-up  CSM  to  post-clean-up 
CSM 
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WE  RE  DONE 
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BUILDING  STRONG 


£El 


Remember  Camp  SLO? 


University  owned 

►  No  access  restrictions 

►  Cattle  grazing 

►  Geotechnical  classes 

►  Camping 

Multiple,  overlapping 
range  fans 

MRS  ~  2,500  acres 


BUILDING  STRONG 


Former  Camp  SLO  Rl  Results 


Farmer  Camp  San  Lula  Obispo 
MRS  01^02  anti  MRS  OB 
Remedial:  Investigation/ Feasibility  Study 
San  Luis  Obispo  County,  Call  Torn  la 
Figure  5-S 

MECanti  MD  Density  MRS  OB 
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Step  1-  Delineate  the  site 


Farmer  Camp  San  Lula  Odlspo 
MRS  01^02  anti  MRS  OB 
Remedial:  Investigation/ Feasibility  Study 
San  Luis  Obispo  County,  Call  Torn  la 
Figure  5-S 

MECanti  MD  Density  MRS  OB 


Legend 
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BUILDING  STRONG 


Depth  (meters  below  ground  surface) 


Step  2-  Develop  RAOs  for  each  (sub)MRS 
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amplitude  response 
detection  limit 
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BUILDING  STRONG 


Step  3-  Pick  one  MRS,  create  your  DQO 


7-step  Process 

■  Step  1 :  Problem  Statement 

■  Step  2:  Goals  of  the  Study 

■  Step  3:  Information  Inputs 

■  Step  4:  Boundaries 

■  Step  5:  Analytic  Approach 

■  Step  6:  Performance  Objectives 

■  Step  7:  Detailed  Plan  to  Obtain  Data 


Instructions 

■  Write  a  detailed  problem  statement 

■  Provide  general  statements  or 
ideas  for  the  other  6  steps 
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BUILDING  STRONG^ 


Post  Remedy-  Scenario  1 


Peak  Model  Sitnal  vi  Peak  Neis-e 

WSm|'  ■  "'i' 


Peak  Film  ourpm  <?i  Model  *  Noise 

vs  PHh  Filter  Output  of  Noise 
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BUILDING  STRONG 


Scenario  1-  High  density  not  completed-  data  gap 


Density  From  Dynamic  TEMTADS 


Target  Count  Estimates: 

Inside  High  Density  area:  3,913 

Outside  High  Density  Area:  14,925 

Total:  18,838 


High  Density  Area 
(>4,000  targets/acre) 
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4230.8 

3974.4 

3717.9 

3461.5 

3205.1 

2948.7 

2692.3 

2435.9 

2179.5 

1923.1 

1666.7 

1410.3 

1153.8 

897.4 

641.0 

384.6 

128.2 

Density 

(targets/acre) 
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Scenario  1-  High  density  not  completed-  data  gap 


Density  From  Dynamic  TEMTADS 


4743.6 

4487.2 

4230.8 

3974.4 

3717.9 

3461.5 

3205.1 

2948.7 

2692.3 

2435.9 

2179.5 

1923.1 

1666.7 

1410.3 

1153.8 
897.4 
641.0 
384.6 
128.2 

Density 

(targets/acre) 


High  Density  Area 
(>4,000  targets/acre 


Target  Count  Estimates: 

Inside  High  Density  area:  3,913 

Outside  High  Density  Area:  14,925 

Total:  18,838 


Questions 

■  Did  we  meet  the  RAO  and  intent  of 
the  ROD? 

■  ROD  requires  explanation  of 
significant  differenced? 

►  What  would  the  ESD  look  like? 

■  LUCs? 

►  How  do  we  implement  on  state  lands? 

►  How  do  we  implement  on  private 
lands? 
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Scenario  2-  High  density  completed-  NO  data  gap 


Density  From  Dynamic  TEMTADS 


4743.6 

4487.2 
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Density 

(targets/acre) 


High  Density  Area 
(>4,000  targets/acre 


Target  Count  Estimates: 

Inside  High  Density  area:  3,913 

Outside  High  Density  Area:  14,925 

Total:  18,838 


Questions 

■  Did  we  meet  the  RAO  and  intent  of 
the  ROD? 

■  ROD  requires  explanation  of 
significant  differenced? 

►  What  would  the  ESD  look  like? 

■  LUCs? 

►  How  do  we  implement  on  state  lands? 

►  How  do  we  implement  on  private 
lands? 
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Scenario  3-  Dig  results  comparison 


3a  3b 


Anomaly/Target  Quantities 

16,202 

Anomaly/Target  Quantities 

16,202 

Amplitude  Response 

Amplitude  Response 

Anomalies  Detected 

Anomalies  Detected 

Advanced  Detection  Analysis 

7,035 

Advanced  Detection  Analysis 

7,035 

Targets  (Potential  TOI 

Targets  (Potential  TOI 

Identified  for  Cued  Analysis) 

Identified  for  Cued  Analysis) 

Targets  Identified  for  Dig  List 

753 

Targets  identified  for  Dig  List 

753 

Can’t  Analyze 

(104) 

Can’t  Analyze 

(104) 

Potential  TOI 

(418) 

Potential  TOI 

(418) 

Calibration,  Threshold 

(231) 

Calibration,  Threshold 

(231) 

Verification  and  Validation 

Verification  and  Validation 

Digs 

Digs 

Dig  Records  in  Database 

1,456 

Dig  Records  in  Database 

1,456 

(more  than  one  item  found 

(more  than  one  item  found 

within  a  0.4  meter  radius 

within  a  0.4  meter  radius 

resulted  in  multiple  records 

resulted  in  multiple  records 

for  an  investigated  location) 

for  an  investigated  location) 

Munitions  Debris  (note  that 

(1,223) 

Munitions  Debris  (note  that 

(1,223) 

some  of  the  individual 

some  of  the  individual 

records  indicate  multiple 

records  indicate  multiple 

pieces  of  frag) 

pieces  of  frag) 

QC  or  QA  Seed 

(94) 

QC  or  QA  Seed 

(94) 

Other  Debris  (Primarily  Scrap 

(62) 

Other  Debris  (Primarily  Scrap 

(62) 

Metal) 

Metal) 

Small  Arms 

(36) 

Small  Arms 

(36) 

No  Contact 

(36) 

No  Contact 

(36) 

Other  (shared  with  adjacent 

(4) 

Other  (shared  with  adjacent 

(4) 

anomaly) 

anomaly) 

UXO 

(!)  ,  ~ 

UXO 

(38) 

puiLLiING  STRONG. 


Scenario  4-  verification/validation 


4a  4b 


Anomaly/Target  Quantities 
Amplitude  Response 
Anomalies  Detected 

16,202 

Advanced  Detection  Analysis 
Targets  (Potential  TOI 
Identified  for  Cued  Analysis) 

7,035 

Targets  Identified  for  Dig  List 

553 

Can’t  Analyze 

(104) 

Potential  TOI 

(418) 

Calibration,  Threshold 
Verification  and  Validation 
Digs 

(31) 

Dig  Records  in  Database 
(more  than  one  item  found 
within  a  0.4  meter  radius 
resulted  in  multiple  records 
for  an  investigated  location) 

1,456 

Munitions  Debris  (note  that 
some  of  the  individual 
records  indicate  multiple 
pieces  of  frag) 

(1,223) 

QC  or  QA  Seed 

(14) 

Other  Debris  (Primarily  Scrap 
Metal) 

(62) 
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